2024-09-12T03:05:27.930 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T03:05:27.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:05:28.051 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901675' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKToeMlZNLUjbnAOLQmVArZy0ahxp0W6RWD0i0mUbdmm4XvgCtGvgNkZqBQab9501OKoUGqxU0UBqJEy+Q0w21o= smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKkoPXxDt1AjcY2VG6zeDo+ZH49neWBRtYENTRm2g+vcB8vQcDb2A0YPUkKOuyO51uORPc0bI6C83yMqoQJRmQ= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8Qw3J+0UWosCh+CesLiAsqnd135PUfW75JTN/Wtjoo/kPHF41fTpfMErCuW2IhF5bvazcsAjNQv/uBIl4Cl3o= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T03:05:28.051 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T03:05:28.052 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T03:05:28.052 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T03:05:28.053 INFO:teuthology.task.internal:Checking packages... 2024-09-12T03:05:28.075 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T03:05:28.075 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T03:05:28.075 INFO:teuthology.packaging:ref: None 2024-09-12T03:05:28.075 INFO:teuthology.packaging:tag: None 2024-09-12T03:05:28.076 INFO:teuthology.packaging:branch: squid 2024-09-12T03:05:28.076 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:05:28.076 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T03:05:28.279 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T03:05:28.281 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T03:05:28.288 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T03:05:28.288 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T03:05:28.332 INFO:teuthology.task.internal:Saving configuration 2024-09-12T03:05:28.354 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T03:05:28.392 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T03:05:28.417 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 03:00:38.749554', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKToeMlZNLUjbnAOLQmVArZy0ahxp0W6RWD0i0mUbdmm4XvgCtGvgNkZqBQab9501OKoUGqxU0UBqJEy+Q0w21o='} 2024-09-12T03:05:28.441 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 03:00:38.748115', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKkoPXxDt1AjcY2VG6zeDo+ZH49neWBRtYENTRm2g+vcB8vQcDb2A0YPUkKOuyO51uORPc0bI6C83yMqoQJRmQ='} 2024-09-12T03:05:28.463 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 03:00:38.746457', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8Qw3J+0UWosCh+CesLiAsqnd135PUfW75JTN/Wtjoo/kPHF41fTpfMErCuW2IhF5bvazcsAjNQv/uBIl4Cl3o='} 2024-09-12T03:05:28.463 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T03:05:28.469 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T03:05:28.469 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T03:05:28.470 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T03:05:28.470 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T03:05:28.578 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f21a5d1bbe0>, signals=[15]) 2024-09-12T03:05:28.578 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T03:05:28.585 INFO:teuthology.task.internal:Opening connections... 2024-09-12T03:05:28.585 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:05:28.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:05:28.664 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-09-12T03:05:28.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:05:28.737 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T03:05:28.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:05:28.812 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T03:05:28.819 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-09-12T03:05:28.837 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-09-12T03:05:28.837 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-09-12T03:05:28.894 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-09-12T03:05:28.894 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-09-12T03:05:28.894 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:05:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:05:28.896 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-09-12T03:05:28.919 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-09-12T03:05:28.938 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-09-12T03:05:28.938 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-09-12T03:05:28.997 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:05:28.998 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:05:28.998 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:05:28.998 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:05:28.998 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:05:28.998 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:05:28.998 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-09-12T03:05:29.019 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-09-12T03:05:29.037 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-09-12T03:05:29.038 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-09-12T03:05:29.095 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:05:29.096 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:05:29.097 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-09-12T03:05:29.119 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T03:05:29.130 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T03:05:29.217 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T03:05:29.217 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:05:29.220 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:05:29.223 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:05:29.241 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T03:05:29.361 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T03:05:29.361 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:05:29.364 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:05:29.366 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:05:29.626 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T03:05:29.702 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T03:05:29.717 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T03:05:29.717 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-12T03:05:29.717 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-12T03:05:29.718 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:05:29.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:05:29.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-12T03:05:29.905 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T03:05:29.905 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-12T03:05:29.905 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-09-12T03:05:29.923 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:05:29.923 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T03:05:29.923 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-12T03:05:29.924 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum upgrade -y linux-firmware 2024-09-12T03:05:32.346 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-12T03:05:34.288 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-12T03:05:39.097 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.5 MB 00:00 2024-09-12T03:05:40.690 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-12T03:05:41.577 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T03:05:47.191 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-12T03:05:49.381 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-09-12T03:05:49.382 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-09-12T03:05:49.382 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-09-12T03:05:49.488 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-12T03:05:49.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T03:05:49.641 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-12T03:05:49.642 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-12T03:05:49.644 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi050.front.sepia.ceph.com to complete... 2024-09-12T03:05:55.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12 03:05:55 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-12T03:05:55.999 INFO:teuthology.orchestra.run.smithi050.stderr:FINISHED --2024-09-12 03:05:55-- 2024-09-12T03:05:55.999 INFO:teuthology.orchestra.run.smithi050.stderr:Total wall clock time: 6.3s 2024-09-12T03:05:55.999 INFO:teuthology.orchestra.run.smithi050.stderr:Downloaded: 1 files, 343M in 6.0s (56.9 MB/s) 2024-09-12T03:05:56.002 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-12T03:05:56.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-12T03:05:58.209 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying... ######################################## 2024-09-12T03:05:58.369 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing... ######################################## 2024-09-12T03:05:58.482 INFO:teuthology.orchestra.run.smithi050.stdout:Updating / installing... 2024-09-12T03:06:06.766 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-12T03:06:46.794 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-12T03:06:46.794 DEBUG:teuthology.orchestra.run.smithi050:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-12T03:06:46.884 INFO:teuthology.orchestra.run.smithi050.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-12T03:06:46.885 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-12T03:06:46.886 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-12T03:06:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-12T03:06:46.888 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-12T03:06:46.889 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-12T03:06:46.890 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-12T03:06:46.891 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-12T03:06:46.892 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-12T03:06:46.893 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-12T03:06:46.894 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-12T03:06:46.895 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-12T03:06:46.896 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-12T03:06:46.897 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-12T03:06:46.898 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-12T03:06:46.899 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-12T03:06:46.900 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-12T03:06:46.901 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-12T03:06:46.902 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-12T03:06:46.903 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-12T03:06:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-12T03:06:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-12T03:06:46.906 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-12T03:06:46.907 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-12T03:06:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-12T03:06:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-12T03:06:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-12T03:06:46.911 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-12T03:06:46.912 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-12T03:06:46.913 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-12T03:06:46.914 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-12T03:06:46.915 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-12T03:06:46.916 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-12T03:06:46.917 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-12T03:06:46.918 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-12T03:06:46.919 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-12T03:06:46.920 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-12T03:06:46.921 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-12T03:06:46.922 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-12T03:06:46.923 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-12T03:06:46.924 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-12T03:06:46.925 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-12T03:06:46.926 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-12T03:06:46.927 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-12T03:06:46.928 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-12T03:06:46.929 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-12T03:06:46.930 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-12T03:06:46.931 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-12T03:06:46.932 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-12T03:06:46.933 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-12T03:06:46.934 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-12T03:06:46.935 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-12T03:06:46.936 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-12T03:06:46.937 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-12T03:06:46.938 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-12T03:06:46.939 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-12T03:06:46.940 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-12T03:06:46.941 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-12T03:06:46.942 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-12T03:06:46.943 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-12T03:06:46.944 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-12T03:06:46.945 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-12T03:06:46.946 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-12T03:06:46.947 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-12T03:06:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-12T03:06:46.949 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-12T03:06:46.950 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-12T03:06:46.951 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-12T03:06:46.952 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-12T03:06:46.953 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-12T03:06:46.954 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-12T03:06:46.955 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-12T03:06:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-12T03:06:46.957 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-12T03:06:46.958 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-12T03:06:46.959 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-12T03:06:46.960 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-12T03:06:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-12T03:06:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-12T03:06:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-12T03:06:46.964 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-12T03:06:46.965 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-12T03:06:46.966 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-12T03:06:46.967 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-12T03:06:46.968 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-12T03:06:46.969 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-12T03:06:46.970 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-12T03:06:46.971 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-12T03:06:46.972 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-12T03:06:46.973 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-12T03:06:46.974 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-12T03:06:46.975 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-12T03:06:46.976 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-12T03:06:46.977 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-12T03:06:46.978 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-12T03:06:46.979 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-12T03:06:46.980 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-12T03:06:46.981 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-12T03:06:46.982 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-12T03:06:46.983 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-12T03:06:46.984 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-12T03:06:46.985 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-12T03:06:46.986 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-12T03:06:46.987 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-12T03:06:46.988 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-12T03:06:46.989 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-12T03:06:46.990 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-12T03:06:46.991 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-12T03:06:46.992 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-12T03:06:46.993 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-12T03:06:46.994 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-12T03:06:46.995 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-12T03:06:46.996 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-12T03:06:46.997 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-12T03:06:46.998 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-12T03:06:46.999 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-12T03:06:47.000 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-12T03:06:47.001 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-12T03:06:47.002 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-12T03:06:47.003 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-12T03:06:47.004 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-12T03:06:47.005 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-12T03:06:47.006 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-12T03:06:47.007 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-12T03:06:47.008 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-12T03:06:47.009 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-12T03:06:47.010 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-12T03:06:47.011 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-12T03:06:47.012 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-12T03:06:47.013 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-12T03:06:47.014 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-12T03:06:47.015 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-12T03:06:47.016 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-12T03:06:47.017 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-12T03:06:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-12T03:06:47.019 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-12T03:06:47.020 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-12T03:06:47.021 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-12T03:06:47.022 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-12T03:06:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-12T03:06:47.024 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-12T03:06:47.025 INFO:teuthology.orchestra.run.smithi050.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T03:06:47.029 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:47.034 DEBUG:teuthology.orchestra.run.smithi050:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-12T03:06:47.056 INFO:teuthology.orchestra.run.smithi050.stdout:postinstall scriptlet (using /bin/sh): 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout:for file in vmlinuz System.map config; do 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: fi 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout:done 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T03:06:47.057 INFO:teuthology.orchestra.run.smithi050.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:47.058 INFO:teuthology.orchestra.run.smithi050.stdout:fi 2024-09-12T03:06:47.059 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-09-12T03:06:47.137 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-09-12T03:06:47.137 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-09-12T03:06:47.137 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-09-12T03:06:47.137 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 82.el9 2024-09-12T03:06:47.137 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-09-12T03:06:47.137 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8274670 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:06:47.138 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:06:47.139 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:06:47.140 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:06:47.140 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:47.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:06:47.741 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-09-12T03:06:49.119 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:06:49.149 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-09-12T03:06:49.151 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-09-12T03:06:49.166 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.AgNiyHvRxQ 2024-09-12T03:06:49.167 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AgNiyHvRxQ 2024-09-12T03:06:49.232 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.AgNiyHvRxQ 2024-09-12T03:06:49.374 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.AgNiyHvRxQ is 6KB 2024-09-12T03:06:49.423 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.AgNiyHvRxQ 2024-09-12T03:06:49.438 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:06:49.500 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:06:49.500 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:06:49.500 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:06:49.500 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-12T03:06:49.502 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:06:49.652 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-09-12T03:07:19.655 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T03:07:19.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-12T03:07:19.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:07:38.177 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-09-12T03:07:47.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-12T03:07:47.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:07:50.241 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-09-12T03:08:02.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-12T03:08:02.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:34.179 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-09-12T03:08:34.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-12T03:08:34.745 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-12T03:08:34.745 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-09-12T03:08:34.780 INFO:teuthology.orchestra.run.smithi050.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:08:34.780 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T03:08:34.781 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:08:34.781 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:08:35.781 DEBUG:teuthology.parallel:result is None 2024-09-12T03:08:35.781 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T03:08:35.790 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T03:08:35.790 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:08:35.794 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:08:35.796 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:08:35.825 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T03:08:35.914 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T03:08:35.979 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T03:08:35.979 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:08:35.983 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:08:35.986 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:08:36.085 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T03:08:36.093 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T03:08:36.093 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:08:36.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:36.110 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:08:36.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:36.128 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:08:36.132 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:08:36.153 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:08:36.213 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:08:36.234 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:08:36.236 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:08:36.254 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:08:36.293 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:08:36.327 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:08:36.330 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T03:08:36.338 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T03:08:36.338 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:08:36.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:08:36.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:08:36.397 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T03:08:36.406 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T03:08:36.406 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:08:36.411 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:08:36.415 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:08:36.441 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:08:36.545 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:08:36.619 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:08:36.685 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:08:36.752 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:08:36.752 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:08:36.807 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:08:36.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:08:36.902 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:08:36.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:08:37.031 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:08:37.031 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:08:37.097 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:08:37.143 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:08:37.208 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:08:37.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:08:37.297 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:08:37.297 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:08:37.364 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-09-12T03:08:37.368 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-09-12T03:08:37.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-09-12T03:08:37.409 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:08:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:08:37.456 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:08:37.892 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T03:08:37.899 INFO:teuthology.task.internal:Starting timer... 2024-09-12T03:08:37.900 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T03:08:38.057 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T03:08:38.144 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T03:08:38.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-09-12T03:08:38.244 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-09-12T03:08:38.247 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-09-12T03:08:38.310 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-09-12T03:08:38.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-09-12T03:08:38.356 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-09-12T03:08:38.358 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T03:08:38.358 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-09-12T03:08:38.401 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-09-12T03:08:38.402 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-09-12T03:08:38.429 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-09-12T03:08:38.429 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-09-12T03:08:38.453 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-09-12T03:08:38.453 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-09-12T03:08:38.454 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:08:38.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:38.512 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:08:38.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:38.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:08:38.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:38.566 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T03:08:38.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:08:38.634 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:08:38.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:08:38.695 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T03:08:38.706 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T03:08:38.719 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T03:08:38.720 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi082.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T03:13:22.205 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-09-12T03:13:22.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-12T03:13:22.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:13:22.294 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-09-12T03:13:22.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-12T03:13:22.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-09-12T03:13:22.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:13:22.490 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-09-12T03:13:22.576 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-09-12T03:13:22.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T03:13:22.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:13:22.650 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-12T03:13:22.732 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T03:13:22.735 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T03:13:22.751 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T03:13:22.751 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:13:22.752 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:13:22.756 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:13:22.756 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:13:22.759 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:13:22.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:13:22.785 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:13:22.800 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:13:22.804 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:13:22.816 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:13:22.820 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:13:22.824 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-09-12T03:13:22.835 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-09-12T03:13:22.845 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-09-12T03:13:22.846 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:13:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:13:22.856 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-09-12T03:13:22.863 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:13:22.870 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:13:22.884 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:13:22.897 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-09-12T03:13:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:13:22.921 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:13:22.921 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-09-12T03:13:22.921 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.921 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.921 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.922 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.922 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-09-12T03:13:22.940 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:13:22.943 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:13:22.943 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-12T03:13:22.943 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.943 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.943 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.944 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:22.951 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:13:22.981 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:13:23.074 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:13:23.080 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:13:23.081 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-09-12T03:13:23.081 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:23.081 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:23.081 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:23.081 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:23.082 INFO:teuthology.run_tasks:Running task install... 2024-09-12T03:13:23.090 DEBUG:teuthology.task.install:project ceph 2024-09-12T03:13:23.090 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T03:13:23.090 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T03:13:23.090 INFO:teuthology.task.install:Using flavor: default 2024-09-12T03:13:23.096 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T03:13:23.096 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T03:13:23.096 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:13:23.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:13:23.099 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:13:23.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:13:23.100 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:13:23.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:13:23.242 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:13:23.242 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:13:23.256 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:13:23.256 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:13:23.265 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:13:23.265 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:13:23.357 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:13:23.357 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:13:23.357 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:13:23.369 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:13:23.369 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:13:23.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:13:23.376 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:13:23.376 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:13:23.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:13:23.396 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:13:23.396 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:13:23.402 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:13:23.402 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:13:23.415 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:13:23.415 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:13:23.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:13:23.468 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:13:23.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:13:23.536 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:13:23.542 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:13:23.591 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:13:23.599 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-09-12T03:13:23.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-09-12T03:13:23.605 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-09-12T03:13:23.607 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-09-12T03:13:23.647 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-09-12T03:13:23.650 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-09-12T03:13:23.826 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-09-12T03:13:23.830 INFO:teuthology.orchestra.run.smithi082.stdout:45 files removed 2024-09-12T03:13:23.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:13:23.857 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:13:23.925 INFO:teuthology.orchestra.run.smithi050.stdout:45 files removed 2024-09-12T03:13:23.970 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:13:24.401 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-12T03:13:24.402 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-12T03:13:24.676 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-12T03:13:24.694 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T03:13:24.794 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-09-12T03:13:24.935 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-12T03:13:24.969 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T03:13:25.087 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T03:13:25.371 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-12T03:13:26.137 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-12T03:13:26.543 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.2 MB 00:01 2024-09-12T03:13:27.511 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.2 MB 00:02 2024-09-12T03:13:28.288 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-12T03:13:30.679 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 20 MB 00:02 2024-09-12T03:13:34.264 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-12T03:13:35.527 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-12T03:13:35.965 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-12T03:13:36.255 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-12T03:13:36.927 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-12T03:13:37.771 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-12T03:13:38.828 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-12T03:13:39.761 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-12T03:13:41.244 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-12T03:13:42.140 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-12T03:13:43.433 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-12T03:13:44.175 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-09-12T03:13:45.937 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:13:45.938 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:13:45.950 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:13:45.951 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:13:45.952 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:13:45.952 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:13:45.954 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:13:45.957 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:13:45.957 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:13:46.027 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-09-12T03:13:46.047 INFO:teuthology.orchestra.run.smithi050.stdout:====================================================================================== 2024-09-12T03:13:46.047 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-09-12T03:13:46.047 INFO:teuthology.orchestra.run.smithi050.stdout:====================================================================================== 2024-09-12T03:13:46.047 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-09-12T03:13:46.047 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:13:46.048 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:13:46.049 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:13:46.050 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:13:46.051 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:13:46.052 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:13:46.053 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:13:46.054 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:13:46.055 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:13:46.056 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:13:46.057 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:13:46.058 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:13:46.059 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout:====================================================================================== 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout:Install 148 Packages 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 8 Packages 2024-09-12T03:13:46.060 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:13:46.061 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 226 M 2024-09-12T03:13:46.061 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-09-12T03:13:46.430 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:13:46.431 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:13:46.444 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:13:46.445 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:13:46.446 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:13:46.447 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:13:46.449 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:13:46.452 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:13:46.453 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:13:46.515 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout:====================================================================================== 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout:====================================================================================== 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:13:46.535 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:13:46.536 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:13:46.537 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-12T03:13:46.538 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:13:46.539 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:13:46.540 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:13:46.541 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:13:46.542 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:13:46.543 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:13:46.544 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:13:46.545 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:13:46.546 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout:====================================================================================== 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout:Install 148 Packages 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 8 Packages 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 226 M 2024-09-12T03:13:46.547 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-12T03:13:47.018 INFO:teuthology.orchestra.run.smithi050.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 40 kB/s | 6.5 kB 00:00 2024-09-12T03:13:47.293 INFO:teuthology.orchestra.run.smithi050.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-12T03:13:47.294 INFO:teuthology.orchestra.run.smithi142.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 35 kB/s | 6.5 kB 00:00 2024-09-12T03:13:47.326 INFO:teuthology.orchestra.run.smithi050.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.8 MB/s | 897 kB 00:00 2024-09-12T03:13:47.377 INFO:teuthology.orchestra.run.smithi050.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.7 MB/s | 145 kB 00:00 2024-09-12T03:13:47.477 INFO:teuthology.orchestra.run.smithi050.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 1.5 MB 00:00 2024-09-12T03:13:47.562 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-12T03:13:47.810 INFO:teuthology.orchestra.run.smithi142.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-09-12T03:13:47.844 INFO:teuthology.orchestra.run.smithi142.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 897 kB 00:00 2024-09-12T03:13:47.852 INFO:teuthology.orchestra.run.smithi050.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-12T03:13:47.877 INFO:teuthology.orchestra.run.smithi142.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-12T03:13:48.011 INFO:teuthology.orchestra.run.smithi050.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.5 MB/s | 2.4 MB 00:00 2024-09-12T03:13:48.128 INFO:teuthology.orchestra.run.smithi142.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.1 MB/s | 1.5 MB 00:00 2024-09-12T03:13:48.361 INFO:teuthology.orchestra.run.smithi050.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 21 MB 00:01 2024-09-12T03:13:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-12T03:13:48.428 INFO:teuthology.orchestra.run.smithi050.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 380 kB/s | 25 kB 00:00 2024-09-12T03:13:48.536 INFO:teuthology.orchestra.run.smithi142.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-09-12T03:13:48.995 INFO:teuthology.orchestra.run.smithi050.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 15 MB/s | 17 MB 00:01 2024-09-12T03:13:49.053 INFO:teuthology.orchestra.run.smithi050.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.9 MB/s | 228 kB 00:00 2024-09-12T03:13:49.154 INFO:teuthology.orchestra.run.smithi050.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 270 kB/s | 27 kB 00:00 2024-09-12T03:13:49.246 INFO:teuthology.orchestra.run.smithi050.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 347 kB/s | 32 kB 00:00 2024-09-12T03:13:49.338 INFO:teuthology.orchestra.run.smithi050.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-12T03:13:49.421 INFO:teuthology.orchestra.run.smithi050.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-12T03:13:49.496 INFO:teuthology.orchestra.run.smithi050.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.7 MB/s | 127 kB 00:00 2024-09-12T03:13:49.614 INFO:teuthology.orchestra.run.smithi050.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.2 MB/s | 506 kB 00:00 2024-09-12T03:13:49.806 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:13:49.806 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:13:49.817 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:13:49.818 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:13:49.819 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:13:49.820 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:13:49.821 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:13:49.824 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:13:49.824 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:13:49.885 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-09-12T03:13:49.908 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================== 2024-09-12T03:13:49.909 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-09-12T03:13:49.909 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================== 2024-09-12T03:13:49.909 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-09-12T03:13:49.909 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:13:49.909 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:13:49.910 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:13:49.910 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:13:49.910 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:13:49.910 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:13:49.910 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:13:49.911 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:13:49.911 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:13:49.911 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:13:49.911 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:13:49.911 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:13:49.911 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:13:49.912 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:13:49.912 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:13:49.912 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:13:49.912 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:13:49.912 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:13:49.913 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:13:49.913 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:13:49.913 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:13:49.913 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:13:49.913 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:13:49.913 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:13:49.914 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:13:49.915 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:13:49.915 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:13:49.915 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:13:49.915 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:13:49.915 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:13:49.915 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:13:49.916 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:13:49.917 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:13:49.918 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:13:49.919 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:13:49.920 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:13:49.921 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:13:49.922 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:13:49.923 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:13:49.924 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:13:49.925 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout:====================================================================================== 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout:Install 148 Packages 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 8 Packages 2024-09-12T03:13:49.926 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:13:49.927 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 226 M 2024-09-12T03:13:49.927 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-09-12T03:13:49.937 INFO:teuthology.orchestra.run.smithi142.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 17 MB 00:01 2024-09-12T03:13:50.003 INFO:teuthology.orchestra.run.smithi142.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 380 kB/s | 25 kB 00:00 2024-09-12T03:13:50.014 INFO:teuthology.orchestra.run.smithi050.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-12T03:13:50.173 INFO:teuthology.orchestra.run.smithi050.stdout:(19/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 4.9 MB/s | 11 MB 00:02 2024-09-12T03:13:50.197 INFO:teuthology.orchestra.run.smithi082.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T03:13:50.197 INFO:teuthology.orchestra.run.smithi050.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 246 kB/s | 45 kB 00:00 2024-09-12T03:13:50.231 INFO:teuthology.orchestra.run.smithi050.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.3 MB/s | 136 kB 00:00 2024-09-12T03:13:50.256 INFO:teuthology.orchestra.run.smithi050.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.7 MB/s | 162 kB 00:00 2024-09-12T03:13:50.290 INFO:teuthology.orchestra.run.smithi050.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T03:13:50.295 INFO:teuthology.orchestra.run.smithi142.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 6.6 MB/s | 21 MB 00:03 2024-09-12T03:13:50.316 INFO:teuthology.orchestra.run.smithi050.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T03:13:50.349 INFO:teuthology.orchestra.run.smithi050.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T03:13:50.391 INFO:teuthology.orchestra.run.smithi050.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-12T03:13:50.421 INFO:teuthology.orchestra.run.smithi142.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.8 MB/s | 228 kB 00:00 2024-09-12T03:13:50.431 INFO:teuthology.orchestra.run.smithi082.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-12T03:13:50.521 INFO:teuthology.orchestra.run.smithi142.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 271 kB/s | 27 kB 00:00 2024-09-12T03:13:50.531 INFO:teuthology.orchestra.run.smithi082.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-12T03:13:50.613 INFO:teuthology.orchestra.run.smithi142.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 346 kB/s | 32 kB 00:00 2024-09-12T03:13:50.781 INFO:teuthology.orchestra.run.smithi082.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 5.4 MB 00:00 2024-09-12T03:13:50.788 INFO:teuthology.orchestra.run.smithi142.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.2 MB/s | 752 kB 00:00 2024-09-12T03:13:50.863 INFO:teuthology.orchestra.run.smithi142.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T03:13:50.915 INFO:teuthology.orchestra.run.smithi082.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 6.2 MB/s | 2.4 MB 00:00 2024-09-12T03:13:50.930 INFO:teuthology.orchestra.run.smithi142.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-12T03:13:50.948 INFO:teuthology.orchestra.run.smithi082.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-12T03:13:51.047 INFO:teuthology.orchestra.run.smithi142.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.2 MB/s | 506 kB 00:00 2024-09-12T03:13:51.133 INFO:teuthology.orchestra.run.smithi050.stdout:(27/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T03:13:51.158 INFO:teuthology.orchestra.run.smithi050.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 223 kB/s | 171 kB 00:00 2024-09-12T03:13:51.225 INFO:teuthology.orchestra.run.smithi050.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 3.5 MB/s | 3.1 MB 00:00 2024-09-12T03:13:51.250 INFO:teuthology.orchestra.run.smithi050.stdout:(30/156): ceph-grafana-dashboards-19.1.1-298.gb 273 kB/s | 32 kB 00:00 2024-09-12T03:13:51.276 INFO:teuthology.orchestra.run.smithi050.stdout:(31/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.2 MB/s | 146 kB 00:00 2024-09-12T03:13:51.376 INFO:teuthology.orchestra.run.smithi050.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T03:13:51.384 INFO:teuthology.orchestra.run.smithi082.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T03:13:51.468 INFO:teuthology.orchestra.run.smithi050.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 539 kB/s | 49 kB 00:00 2024-09-12T03:13:51.576 INFO:teuthology.orchestra.run.smithi050.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 152 kB/s | 16 kB 00:00 2024-09-12T03:13:52.064 INFO:teuthology.orchestra.run.smithi142.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.3 MB/s | 5.3 MB 00:01 2024-09-12T03:13:52.139 INFO:teuthology.orchestra.run.smithi142.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 601 kB/s | 45 kB 00:00 2024-09-12T03:13:52.185 INFO:teuthology.orchestra.run.smithi050.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 7.9 MB/s | 7.4 MB 00:00 2024-09-12T03:13:52.215 INFO:teuthology.orchestra.run.smithi142.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-12T03:13:52.244 INFO:teuthology.orchestra.run.smithi050.stdout:(36/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 3.5 MB/s | 3.6 MB 00:01 2024-09-12T03:13:52.269 INFO:teuthology.orchestra.run.smithi050.stdout:(37/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 412 kB/s | 285 kB 00:00 2024-09-12T03:13:52.282 INFO:teuthology.orchestra.run.smithi142.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T03:13:52.303 INFO:teuthology.orchestra.run.smithi050.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 312 kB/s | 18 kB 00:00 2024-09-12T03:13:52.336 INFO:teuthology.orchestra.run.smithi050.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-12T03:13:52.365 INFO:teuthology.orchestra.run.smithi082.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 9.2 MB/s | 21 MB 00:02 2024-09-12T03:13:52.390 INFO:teuthology.orchestra.run.smithi142.stdout:(22/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 2.8 MB/s | 11 MB 00:03 2024-09-12T03:13:52.403 INFO:teuthology.orchestra.run.smithi050.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 597 kB/s | 80 kB 00:00 2024-09-12T03:13:52.415 INFO:teuthology.orchestra.run.smithi142.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.3 MB/s | 321 kB 00:00 2024-09-12T03:13:52.466 INFO:teuthology.orchestra.run.smithi142.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T03:13:52.487 INFO:teuthology.orchestra.run.smithi050.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-12T03:13:52.499 INFO:teuthology.orchestra.run.smithi082.stdout:(9/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 9.5 MB/s | 11 MB 00:01 2024-09-12T03:13:52.499 INFO:teuthology.orchestra.run.smithi142.stdout:(25/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.7 MB/s | 301 kB 00:00 2024-09-12T03:13:52.512 INFO:teuthology.orchestra.run.smithi050.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-12T03:13:52.524 INFO:teuthology.orchestra.run.smithi082.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 159 kB/s | 25 kB 00:00 2024-09-12T03:13:52.533 INFO:teuthology.orchestra.run.smithi142.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-12T03:13:52.545 INFO:teuthology.orchestra.run.smithi050.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-12T03:13:52.571 INFO:teuthology.orchestra.run.smithi050.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T03:13:52.592 INFO:teuthology.orchestra.run.smithi142.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T03:13:52.596 INFO:teuthology.orchestra.run.smithi050.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T03:13:52.621 INFO:teuthology.orchestra.run.smithi050.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T03:13:52.633 INFO:teuthology.orchestra.run.smithi142.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 768 kB/s | 32 kB 00:00 2024-09-12T03:13:52.647 INFO:teuthology.orchestra.run.smithi050.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T03:13:52.691 INFO:teuthology.orchestra.run.smithi082.stdout:(11/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 9.7 MB/s | 17 MB 00:01 2024-09-12T03:13:52.692 INFO:teuthology.orchestra.run.smithi142.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-12T03:13:52.697 INFO:teuthology.orchestra.run.smithi050.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-09-12T03:13:52.716 INFO:teuthology.orchestra.run.smithi082.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.2 MB/s | 228 kB 00:00 2024-09-12T03:13:52.750 INFO:teuthology.orchestra.run.smithi082.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 463 kB/s | 27 kB 00:00 2024-09-12T03:13:52.775 INFO:teuthology.orchestra.run.smithi082.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 543 kB/s | 32 kB 00:00 2024-09-12T03:13:52.842 INFO:teuthology.orchestra.run.smithi082.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T03:13:52.864 INFO:teuthology.orchestra.run.smithi050.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-12T03:13:52.876 INFO:teuthology.orchestra.run.smithi082.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.9 MB/s | 752 kB 00:00 2024-09-12T03:13:52.918 INFO:teuthology.orchestra.run.smithi082.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T03:13:52.931 INFO:teuthology.orchestra.run.smithi050.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-12T03:13:52.943 INFO:teuthology.orchestra.run.smithi082.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T03:13:52.956 INFO:teuthology.orchestra.run.smithi050.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 237 kB/s | 22 kB 00:00 2024-09-12T03:13:52.989 INFO:teuthology.orchestra.run.smithi050.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-12T03:13:53.010 INFO:teuthology.orchestra.run.smithi082.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 674 kB/s | 45 kB 00:00 2024-09-12T03:13:53.015 INFO:teuthology.orchestra.run.smithi050.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-12T03:13:53.040 INFO:teuthology.orchestra.run.smithi050.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T03:13:53.065 INFO:teuthology.orchestra.run.smithi050.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T03:13:53.076 INFO:teuthology.orchestra.run.smithi142.stdout:(30/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 9.3 MB/s | 3.6 MB 00:00 2024-09-12T03:13:53.077 INFO:teuthology.orchestra.run.smithi082.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T03:13:53.090 INFO:teuthology.orchestra.run.smithi050.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-12T03:13:53.126 INFO:teuthology.orchestra.run.smithi142.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 5.0 MB/s | 3.1 MB 00:00 2024-09-12T03:13:53.141 INFO:teuthology.orchestra.run.smithi050.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-12T03:13:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T03:13:53.166 INFO:teuthology.orchestra.run.smithi050.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-12T03:13:53.219 INFO:teuthology.orchestra.run.smithi082.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-12T03:13:53.233 INFO:teuthology.orchestra.run.smithi050.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-12T03:13:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.3 MB/s | 249 kB 00:00 2024-09-12T03:13:53.283 INFO:teuthology.orchestra.run.smithi050.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-12T03:13:53.285 INFO:teuthology.orchestra.run.smithi142.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 983 kB/s | 49 kB 00:00 2024-09-12T03:13:53.294 INFO:teuthology.orchestra.run.smithi082.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-12T03:13:53.309 INFO:teuthology.orchestra.run.smithi050.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 326 kB/s | 24 kB 00:00 2024-09-12T03:13:53.320 INFO:teuthology.orchestra.run.smithi082.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.9 MB/s | 301 kB 00:00 2024-09-12T03:13:53.327 INFO:teuthology.orchestra.run.smithi142.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 391 kB/s | 16 kB 00:00 2024-09-12T03:13:53.333 INFO:teuthology.orchestra.run.smithi050.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-12T03:13:53.362 INFO:teuthology.orchestra.run.smithi082.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 100 kB 00:00 2024-09-12T03:13:53.384 INFO:teuthology.orchestra.run.smithi050.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T03:13:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-12T03:13:53.419 INFO:teuthology.orchestra.run.smithi142.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.0 MB/s | 285 kB 00:00 2024-09-12T03:13:53.426 INFO:teuthology.orchestra.run.smithi050.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 930 kB/s | 38 kB 00:00 2024-09-12T03:13:53.454 INFO:teuthology.orchestra.run.smithi082.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-12T03:13:53.476 INFO:teuthology.orchestra.run.smithi050.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-12T03:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 480 kB/s | 32 kB 00:00 2024-09-12T03:13:53.588 INFO:teuthology.orchestra.run.smithi082.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.1 MB/s | 146 kB 00:00 2024-09-12T03:13:53.628 INFO:teuthology.orchestra.run.smithi142.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.6 MB/s | 765 kB 00:00 2024-09-12T03:13:53.638 INFO:teuthology.orchestra.run.smithi082.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-12T03:13:53.678 INFO:teuthology.orchestra.run.smithi142.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 365 kB/s | 18 kB 00:00 2024-09-12T03:13:53.843 INFO:teuthology.orchestra.run.smithi050.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-09-12T03:13:53.912 INFO:teuthology.orchestra.run.smithi142.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.9 MB/s | 7.4 MB 00:00 2024-09-12T03:13:53.966 INFO:teuthology.orchestra.run.smithi082.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 9.4 MB/s | 3.6 MB 00:00 2024-09-12T03:13:54.033 INFO:teuthology.orchestra.run.smithi082.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T03:13:54.150 INFO:teuthology.orchestra.run.smithi082.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-12T03:13:54.175 INFO:teuthology.orchestra.run.smithi082.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 347 kB/s | 49 kB 00:00 2024-09-12T03:13:54.201 INFO:teuthology.orchestra.run.smithi082.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-12T03:13:54.210 INFO:teuthology.orchestra.run.smithi050.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-09-12T03:13:54.226 INFO:teuthology.orchestra.run.smithi082.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-12T03:13:54.276 INFO:teuthology.orchestra.run.smithi082.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-12T03:13:54.310 INFO:teuthology.orchestra.run.smithi082.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 218 kB/s | 18 kB 00:00 2024-09-12T03:13:54.437 INFO:teuthology.orchestra.run.smithi142.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 11 MB/s | 49 MB 00:04 2024-09-12T03:13:54.463 INFO:teuthology.orchestra.run.smithi142.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 102 kB/s | 80 kB 00:00 2024-09-12T03:13:54.719 INFO:teuthology.orchestra.run.smithi050.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.7 MB/s | 910 kB 00:00 2024-09-12T03:13:54.785 INFO:teuthology.orchestra.run.smithi082.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 157 kB/s | 80 kB 00:00 2024-09-12T03:13:54.844 INFO:teuthology.orchestra.run.smithi050.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T03:13:54.846 INFO:teuthology.orchestra.run.smithi142.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 661 kB/s | 253 kB 00:00 2024-09-12T03:13:55.038 INFO:teuthology.orchestra.run.smithi142.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 318 kB/s | 191 kB 00:00 2024-09-12T03:13:55.248 INFO:teuthology.orchestra.run.smithi142.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-09-12T03:13:55.252 INFO:teuthology.orchestra.run.smithi082.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T03:13:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 389 kB/s | 191 kB 00:00 2024-09-12T03:13:55.280 INFO:teuthology.orchestra.run.smithi142.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 440 kB/s | 106 kB 00:00 2024-09-12T03:13:55.305 INFO:teuthology.orchestra.run.smithi142.stdout:(45/156): libgfortran-11.5.0-2.el9.x86_64.rpm 575 kB/s | 801 kB 00:01 2024-09-12T03:13:55.364 INFO:teuthology.orchestra.run.smithi142.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-12T03:13:55.369 INFO:teuthology.orchestra.run.smithi082.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 756 kB/s | 801 kB 00:01 2024-09-12T03:13:55.390 INFO:teuthology.orchestra.run.smithi142.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-12T03:13:55.423 INFO:teuthology.orchestra.run.smithi142.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-12T03:13:55.427 INFO:teuthology.orchestra.run.smithi050.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 4.4 MB 00:01 2024-09-12T03:13:55.486 INFO:teuthology.orchestra.run.smithi082.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 106 kB 00:00 2024-09-12T03:13:55.494 INFO:teuthology.orchestra.run.smithi050.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.4 MB/s | 4.9 MB 00:02 2024-09-12T03:13:55.573 INFO:teuthology.orchestra.run.smithi142.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 713 kB/s | 107 kB 00:00 2024-09-12T03:13:55.655 INFO:teuthology.orchestra.run.smithi082.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-12T03:13:55.655 INFO:teuthology.orchestra.run.smithi050.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-12T03:13:55.655 INFO:teuthology.orchestra.run.smithi142.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-12T03:13:55.666 INFO:teuthology.orchestra.run.smithi142.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 235 kB/s | 22 kB 00:00 2024-09-12T03:13:55.699 INFO:teuthology.orchestra.run.smithi142.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-12T03:13:55.703 INFO:teuthology.orchestra.run.smithi082.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi142.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-12T03:13:55.750 INFO:teuthology.orchestra.run.smithi142.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T03:13:55.783 INFO:teuthology.orchestra.run.smithi142.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-12T03:13:55.800 INFO:teuthology.orchestra.run.smithi142.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-12T03:13:55.803 INFO:teuthology.orchestra.run.smithi082.stdout:(46/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 459 kB/s | 253 kB 00:00 2024-09-12T03:13:55.829 INFO:teuthology.orchestra.run.smithi082.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-12T03:13:55.837 INFO:teuthology.orchestra.run.smithi050.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 5.8 MB/s | 5.8 MB 00:00 2024-09-12T03:13:55.842 INFO:teuthology.orchestra.run.smithi142.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T03:13:55.862 INFO:teuthology.orchestra.run.smithi082.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-09-12T03:13:55.867 INFO:teuthology.orchestra.run.smithi142.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 678 kB/s | 45 kB 00:00 2024-09-12T03:13:55.870 INFO:teuthology.orchestra.run.smithi050.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.3 MB/s | 1.0 MB 00:00 2024-09-12T03:13:55.895 INFO:teuthology.orchestra.run.smithi050.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T03:13:55.929 INFO:teuthology.orchestra.run.smithi050.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T03:13:55.954 INFO:teuthology.orchestra.run.smithi050.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-12T03:13:55.980 INFO:teuthology.orchestra.run.smithi050.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-12T03:13:55.984 INFO:teuthology.orchestra.run.smithi142.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-12T03:13:56.005 INFO:teuthology.orchestra.run.smithi050.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-12T03:13:56.010 INFO:teuthology.orchestra.run.smithi142.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-12T03:13:56.027 INFO:teuthology.orchestra.run.smithi142.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 585 kB/s | 24 kB 00:00 2024-09-12T03:13:56.029 INFO:teuthology.orchestra.run.smithi082.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-12T03:13:56.038 INFO:teuthology.orchestra.run.smithi050.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T03:13:56.052 INFO:teuthology.orchestra.run.smithi142.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-12T03:13:56.064 INFO:teuthology.orchestra.run.smithi050.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-12T03:13:56.079 INFO:teuthology.orchestra.run.smithi082.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-09-12T03:13:56.105 INFO:teuthology.orchestra.run.smithi082.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-12T03:13:56.127 INFO:teuthology.orchestra.run.smithi142.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-09-12T03:13:56.130 INFO:teuthology.orchestra.run.smithi082.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-12T03:13:56.147 INFO:teuthology.orchestra.run.smithi082.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 777 kB/s | 33 kB 00:00 2024-09-12T03:13:56.156 INFO:teuthology.orchestra.run.smithi050.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-12T03:13:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T03:13:56.178 INFO:teuthology.orchestra.run.smithi142.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-12T03:13:56.181 INFO:teuthology.orchestra.run.smithi050.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-12T03:13:56.189 INFO:teuthology.orchestra.run.smithi082.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-12T03:13:56.214 INFO:teuthology.orchestra.run.smithi082.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-12T03:13:56.215 INFO:teuthology.orchestra.run.smithi050.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T03:13:56.231 INFO:teuthology.orchestra.run.smithi082.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-12T03:13:56.248 INFO:teuthology.orchestra.run.smithi050.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-12T03:13:56.274 INFO:teuthology.orchestra.run.smithi050.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-12T03:13:56.274 INFO:teuthology.orchestra.run.smithi082.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T03:13:56.299 INFO:teuthology.orchestra.run.smithi082.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-09-12T03:13:56.340 INFO:teuthology.orchestra.run.smithi050.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-12T03:13:56.357 INFO:teuthology.orchestra.run.smithi082.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-12T03:13:56.366 INFO:teuthology.orchestra.run.smithi050.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-12T03:13:56.391 INFO:teuthology.orchestra.run.smithi082.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-12T03:13:56.399 INFO:teuthology.orchestra.run.smithi050.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-12T03:13:56.408 INFO:teuthology.orchestra.run.smithi082.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-12T03:13:56.411 INFO:teuthology.orchestra.run.smithi142.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-09-12T03:13:56.424 INFO:teuthology.orchestra.run.smithi050.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-09-12T03:13:56.441 INFO:teuthology.orchestra.run.smithi082.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-12T03:13:56.450 INFO:teuthology.orchestra.run.smithi050.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:13:56.458 INFO:teuthology.orchestra.run.smithi082.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-09-12T03:13:56.483 INFO:teuthology.orchestra.run.smithi082.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-12T03:13:56.525 INFO:teuthology.orchestra.run.smithi050.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-12T03:13:56.525 INFO:teuthology.orchestra.run.smithi082.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T03:13:56.550 INFO:teuthology.orchestra.run.smithi050.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-12T03:13:56.551 INFO:teuthology.orchestra.run.smithi082.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 418 kB/s | 38 kB 00:00 2024-09-12T03:13:56.575 INFO:teuthology.orchestra.run.smithi050.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-12T03:13:56.609 INFO:teuthology.orchestra.run.smithi050.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T03:13:56.634 INFO:teuthology.orchestra.run.smithi050.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-12T03:13:56.684 INFO:teuthology.orchestra.run.smithi050.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 633 kB/s | 47 kB 00:00 2024-09-12T03:13:56.718 INFO:teuthology.orchestra.run.smithi050.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-12T03:13:56.734 INFO:teuthology.orchestra.run.smithi082.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-12T03:13:56.768 INFO:teuthology.orchestra.run.smithi050.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-09-12T03:13:56.794 INFO:teuthology.orchestra.run.smithi050.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-12T03:13:56.836 INFO:teuthology.orchestra.run.smithi050.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 947 kB/s | 64 kB 00:00 2024-09-12T03:13:56.851 INFO:teuthology.orchestra.run.smithi082.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-12T03:13:56.861 INFO:teuthology.orchestra.run.smithi050.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-09-12T03:13:56.885 INFO:teuthology.orchestra.run.smithi082.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 5.9 MB/s | 910 kB 00:00 2024-09-12T03:13:56.910 INFO:teuthology.orchestra.run.smithi082.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T03:13:56.920 INFO:teuthology.orchestra.run.smithi142.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-09-12T03:13:56.986 INFO:teuthology.orchestra.run.smithi082.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-12T03:13:57.044 INFO:teuthology.orchestra.run.smithi082.stdout:(72/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-09-12T03:13:57.045 INFO:teuthology.orchestra.run.smithi050.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-12T03:13:57.069 INFO:teuthology.orchestra.run.smithi082.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-12T03:13:57.103 INFO:teuthology.orchestra.run.smithi050.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-12T03:13:57.119 INFO:teuthology.orchestra.run.smithi082.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T03:13:57.128 INFO:teuthology.orchestra.run.smithi142.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-09-12T03:13:57.137 INFO:teuthology.orchestra.run.smithi050.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-12T03:13:57.154 INFO:teuthology.orchestra.run.smithi050.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-12T03:13:57.178 INFO:teuthology.orchestra.run.smithi082.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T03:13:57.195 INFO:teuthology.orchestra.run.smithi142.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T03:13:57.254 INFO:teuthology.orchestra.run.smithi142.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 2.7 MB/s | 910 kB 00:00 2024-09-12T03:13:57.271 INFO:teuthology.orchestra.run.smithi050.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-12T03:13:57.287 INFO:teuthology.orchestra.run.smithi082.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-09-12T03:13:57.288 INFO:teuthology.orchestra.run.smithi050.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 49 kB/s | 22 kB 00:00 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi082.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi050.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi050.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 4.4 MB/s | 67 kB 00:00 2024-09-12T03:13:57.346 INFO:teuthology.orchestra.run.smithi082.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-12T03:13:57.363 INFO:teuthology.orchestra.run.smithi050.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-09-12T03:13:57.371 INFO:teuthology.orchestra.run.smithi082.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-09-12T03:13:57.389 INFO:teuthology.orchestra.run.smithi050.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-12T03:13:57.404 INFO:teuthology.orchestra.run.smithi082.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T03:13:57.430 INFO:teuthology.orchestra.run.smithi082.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-12T03:13:57.430 INFO:teuthology.orchestra.run.smithi050.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T03:13:57.456 INFO:teuthology.orchestra.run.smithi050.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:13:57.489 INFO:teuthology.orchestra.run.smithi050.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-12T03:13:57.505 INFO:teuthology.orchestra.run.smithi082.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-12T03:13:57.515 INFO:teuthology.orchestra.run.smithi050.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-09-12T03:13:57.530 INFO:teuthology.orchestra.run.smithi082.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-12T03:13:57.550 INFO:teuthology.orchestra.run.smithi050.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 236 kB/s | 14 kB 00:00 2024-09-12T03:13:57.554 INFO:teuthology.orchestra.run.smithi142.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-09-12T03:13:57.564 INFO:teuthology.orchestra.run.smithi082.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T03:13:57.575 INFO:teuthology.orchestra.run.smithi050.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-09-12T03:13:57.609 INFO:teuthology.orchestra.run.smithi050.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T03:13:57.613 INFO:teuthology.orchestra.run.smithi142.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T03:13:57.614 INFO:teuthology.orchestra.run.smithi082.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 923 kB/s | 77 kB 00:00 2024-09-12T03:13:57.631 INFO:teuthology.orchestra.run.smithi082.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-09-12T03:13:57.633 INFO:teuthology.orchestra.run.smithi050.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T03:13:57.659 INFO:teuthology.orchestra.run.smithi050.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T03:13:57.693 INFO:teuthology.orchestra.run.smithi050.stdout:(122/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-12T03:13:57.706 INFO:teuthology.orchestra.run.smithi082.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-12T03:13:57.732 INFO:teuthology.orchestra.run.smithi082.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-12T03:13:57.735 INFO:teuthology.orchestra.run.smithi050.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-12T03:13:57.738 INFO:teuthology.orchestra.run.smithi142.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.1 MB/s | 4.9 MB 00:01 2024-09-12T03:13:57.760 INFO:teuthology.orchestra.run.smithi050.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-09-12T03:13:57.765 INFO:teuthology.orchestra.run.smithi082.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-09-12T03:13:57.783 INFO:teuthology.orchestra.run.smithi082.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T03:13:57.785 INFO:teuthology.orchestra.run.smithi050.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 369 kB/s | 18 kB 00:00 2024-09-12T03:13:57.789 INFO:teuthology.orchestra.run.smithi142.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T03:13:57.808 INFO:teuthology.orchestra.run.smithi082.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-12T03:13:57.810 INFO:teuthology.orchestra.run.smithi050.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-12T03:13:57.825 INFO:teuthology.orchestra.run.smithi082.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 912 kB/s | 54 kB 00:00 2024-09-12T03:13:57.835 INFO:teuthology.orchestra.run.smithi050.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T03:13:57.858 INFO:teuthology.orchestra.run.smithi082.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-12T03:13:57.861 INFO:teuthology.orchestra.run.smithi050.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-12T03:13:57.883 INFO:teuthology.orchestra.run.smithi082.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 752 kB/s | 44 kB 00:00 2024-09-12T03:13:57.886 INFO:teuthology.orchestra.run.smithi050.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:13:57.889 INFO:teuthology.orchestra.run.smithi142.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-12T03:13:57.912 INFO:teuthology.orchestra.run.smithi050.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:13:57.917 INFO:teuthology.orchestra.run.smithi082.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T03:13:57.937 INFO:teuthology.orchestra.run.smithi050.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T03:13:57.942 INFO:teuthology.orchestra.run.smithi082.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-12T03:13:57.970 INFO:teuthology.orchestra.run.smithi050.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T03:13:57.976 INFO:teuthology.orchestra.run.smithi082.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-12T03:13:57.981 INFO:teuthology.orchestra.run.smithi142.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-12T03:13:57.996 INFO:teuthology.orchestra.run.smithi050.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-12T03:13:58.001 INFO:teuthology.orchestra.run.smithi082.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T03:13:58.013 INFO:teuthology.orchestra.run.smithi050.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 5.3 MB/s | 90 kB 00:00 2024-09-12T03:13:58.035 INFO:teuthology.orchestra.run.smithi082.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 984 kB/s | 58 kB 00:00 2024-09-12T03:13:58.038 INFO:teuthology.orchestra.run.smithi050.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:13:58.039 INFO:teuthology.orchestra.run.smithi142.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-09-12T03:13:58.060 INFO:teuthology.orchestra.run.smithi082.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T03:13:58.063 INFO:teuthology.orchestra.run.smithi050.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-12T03:13:58.094 INFO:teuthology.orchestra.run.smithi082.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T03:13:58.097 INFO:teuthology.orchestra.run.smithi050.stdout:(137/156): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-09-12T03:13:58.098 INFO:teuthology.orchestra.run.smithi142.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-09-12T03:13:58.119 INFO:teuthology.orchestra.run.smithi082.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-12T03:13:58.122 INFO:teuthology.orchestra.run.smithi050.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:13:58.147 INFO:teuthology.orchestra.run.smithi050.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-12T03:13:58.173 INFO:teuthology.orchestra.run.smithi050.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T03:13:58.173 INFO:teuthology.orchestra.run.smithi142.stdout:(79/156): postgresql-server-13.16-1.el9.x86_64. 5.9 MB/s | 5.8 MB 00:00 2024-09-12T03:13:58.198 INFO:teuthology.orchestra.run.smithi050.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T03:13:58.223 INFO:teuthology.orchestra.run.smithi050.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:13:58.224 INFO:teuthology.orchestra.run.smithi142.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-12T03:13:58.249 INFO:teuthology.orchestra.run.smithi050.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T03:13:58.249 INFO:teuthology.orchestra.run.smithi142.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-09-12T03:13:58.290 INFO:teuthology.orchestra.run.smithi050.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-12T03:13:58.294 INFO:teuthology.orchestra.run.smithi082.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-09-12T03:13:58.307 INFO:teuthology.orchestra.run.smithi050.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-12T03:13:58.332 INFO:teuthology.orchestra.run.smithi142.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-12T03:13:58.341 INFO:teuthology.orchestra.run.smithi050.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-12T03:13:58.391 INFO:teuthology.orchestra.run.smithi142.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T03:13:58.433 INFO:teuthology.orchestra.run.smithi050.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-12T03:13:58.450 INFO:teuthology.orchestra.run.smithi142.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T03:13:58.461 INFO:teuthology.orchestra.run.smithi082.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-12T03:13:58.492 INFO:teuthology.orchestra.run.smithi050.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-12T03:13:58.500 INFO:teuthology.orchestra.run.smithi142.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 849 kB/s | 42 kB 00:00 2024-09-12T03:13:58.511 INFO:teuthology.orchestra.run.smithi082.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-09-12T03:13:58.537 INFO:teuthology.orchestra.run.smithi082.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 832 kB/s | 19 kB 00:00 2024-09-12T03:13:58.562 INFO:teuthology.orchestra.run.smithi082.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.0 MB/s | 308 kB 00:00 2024-09-12T03:13:58.584 INFO:teuthology.orchestra.run.smithi142.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-12T03:13:58.587 INFO:teuthology.orchestra.run.smithi082.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T03:13:58.613 INFO:teuthology.orchestra.run.smithi082.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:13:58.638 INFO:teuthology.orchestra.run.smithi082.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T03:13:58.642 INFO:teuthology.orchestra.run.smithi142.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-12T03:13:58.650 INFO:teuthology.orchestra.run.smithi050.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-12T03:13:58.663 INFO:teuthology.orchestra.run.smithi082.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T03:13:58.705 INFO:teuthology.orchestra.run.smithi082.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-12T03:13:58.726 INFO:teuthology.orchestra.run.smithi142.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-12T03:13:58.748 INFO:teuthology.orchestra.run.smithi082.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T03:13:58.776 INFO:teuthology.orchestra.run.smithi142.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T03:13:58.781 INFO:teuthology.orchestra.run.smithi082.stdout:(114/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-09-12T03:13:58.806 INFO:teuthology.orchestra.run.smithi082.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-09-12T03:13:58.817 INFO:teuthology.orchestra.run.smithi050.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.2 MB/s | 3.2 MB 00:00 2024-09-12T03:13:58.826 INFO:teuthology.orchestra.run.smithi142.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:13:58.839 INFO:teuthology.orchestra.run.smithi082.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 532 kB/s | 14 kB 00:00 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi082.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 7.2 MB/s | 172 kB 00:00 2024-09-12T03:13:58.890 INFO:teuthology.orchestra.run.smithi082.stdout:(118/156): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-09-12T03:13:58.916 INFO:teuthology.orchestra.run.smithi082.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-09-12T03:13:58.920 INFO:teuthology.orchestra.run.smithi050.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 5.6 MB/s | 2.3 MB 00:00 2024-09-12T03:13:58.941 INFO:teuthology.orchestra.run.smithi082.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T03:13:58.966 INFO:teuthology.orchestra.run.smithi082.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-12T03:13:58.967 INFO:teuthology.orchestra.run.smithi050.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.2 MB/s | 249 kB 00:00 2024-09-12T03:13:59.000 INFO:teuthology.orchestra.run.smithi082.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T03:13:59.017 INFO:teuthology.orchestra.run.smithi082.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-12T03:13:59.018 INFO:teuthology.orchestra.run.smithi050.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.9 MB/s | 1.8 MB 00:00 2024-09-12T03:13:59.042 INFO:teuthology.orchestra.run.smithi082.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-12T03:13:59.043 INFO:teuthology.orchestra.run.smithi050.stdout:(154/156): NetworkManager-team-1.49.90-1.el9.x8 181 kB/s | 41 kB 00:00 2024-09-12T03:13:59.059 INFO:teuthology.orchestra.run.smithi082.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-12T03:13:59.076 INFO:teuthology.orchestra.run.smithi082.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-12T03:13:59.093 INFO:teuthology.orchestra.run.smithi082.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-12T03:13:59.094 INFO:teuthology.orchestra.run.smithi142.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-09-12T03:13:59.094 INFO:teuthology.orchestra.run.smithi050.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-12T03:13:59.144 INFO:teuthology.orchestra.run.smithi050.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-12T03:13:59.150 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:13:59.150 INFO:teuthology.orchestra.run.smithi050.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-12T03:13:59.152 INFO:teuthology.orchestra.run.smithi142.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 717 kB/s | 42 kB 00:00 2024-09-12T03:13:59.160 INFO:teuthology.orchestra.run.smithi082.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-12T03:13:59.185 INFO:teuthology.orchestra.run.smithi082.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:13:59.202 INFO:teuthology.orchestra.run.smithi142.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-12T03:13:59.219 INFO:teuthology.orchestra.run.smithi082.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-12T03:13:59.244 INFO:teuthology.orchestra.run.smithi082.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T03:13:59.261 INFO:teuthology.orchestra.run.smithi142.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T03:13:59.278 INFO:teuthology.orchestra.run.smithi082.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-12T03:13:59.303 INFO:teuthology.orchestra.run.smithi082.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-12T03:13:59.319 INFO:teuthology.orchestra.run.smithi142.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-09-12T03:13:59.328 INFO:teuthology.orchestra.run.smithi082.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-12T03:13:59.354 INFO:teuthology.orchestra.run.smithi082.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T03:13:59.379 INFO:teuthology.orchestra.run.smithi082.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T03:13:59.395 INFO:teuthology.orchestra.run.smithi142.stdout:(96/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-09-12T03:13:59.404 INFO:teuthology.orchestra.run.smithi082.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-12T03:13:59.420 INFO:teuthology.orchestra.run.smithi142.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-12T03:13:59.479 INFO:teuthology.orchestra.run.smithi142.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-12T03:13:59.488 INFO:teuthology.orchestra.run.smithi082.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 711 kB/s | 59 kB 00:00 2024-09-12T03:13:59.514 INFO:teuthology.orchestra.run.smithi082.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-12T03:13:59.529 INFO:teuthology.orchestra.run.smithi142.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-12T03:13:59.538 INFO:teuthology.orchestra.run.smithi082.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 707 kB/s | 36 kB 00:00 2024-09-12T03:13:59.554 INFO:teuthology.orchestra.run.smithi142.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-12T03:13:59.571 INFO:teuthology.orchestra.run.smithi142.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-12T03:13:59.596 INFO:teuthology.orchestra.run.smithi142.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-12T03:13:59.597 INFO:teuthology.orchestra.run.smithi082.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 892 kB/s | 74 kB 00:00 2024-09-12T03:13:59.655 INFO:teuthology.orchestra.run.smithi142.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-12T03:13:59.656 INFO:teuthology.orchestra.run.smithi082.stdout:(142/156): libarrow-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-12T03:13:59.681 INFO:teuthology.orchestra.run.smithi082.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-09-12T03:13:59.706 INFO:teuthology.orchestra.run.smithi082.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-09-12T03:13:59.714 INFO:teuthology.orchestra.run.smithi142.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-12T03:13:59.732 INFO:teuthology.orchestra.run.smithi082.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-12T03:13:59.756 INFO:teuthology.orchestra.run.smithi142.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-12T03:13:59.768 INFO:teuthology.orchestra.run.smithi082.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T03:13:59.791 INFO:teuthology.orchestra.run.smithi082.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-12T03:13:59.797 INFO:teuthology.orchestra.run.smithi142.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-09-12T03:13:59.833 INFO:teuthology.orchestra.run.smithi082.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T03:13:59.839 INFO:teuthology.orchestra.run.smithi142.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-12T03:13:59.881 INFO:teuthology.orchestra.run.smithi142.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 629 kB/s | 26 kB 00:00 2024-09-12T03:13:59.915 INFO:teuthology.orchestra.run.smithi142.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-12T03:13:59.941 INFO:teuthology.orchestra.run.smithi142.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T03:13:59.974 INFO:teuthology.orchestra.run.smithi142.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-12T03:14:00.033 INFO:teuthology.orchestra.run.smithi082.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-12T03:14:00.041 INFO:teuthology.orchestra.run.smithi142.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-12T03:14:00.083 INFO:teuthology.orchestra.run.smithi142.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-09-12T03:14:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-12T03:14:00.117 INFO:teuthology.orchestra.run.smithi082.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-12T03:14:00.133 INFO:teuthology.orchestra.run.smithi142.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-09-12T03:14:00.159 INFO:teuthology.orchestra.run.smithi142.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-09-12T03:14:00.185 INFO:teuthology.orchestra.run.smithi142.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-12T03:14:00.317 INFO:teuthology.orchestra.run.smithi082.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 204 kB/s | 41 kB 00:00 2024-09-12T03:14:00.403 INFO:teuthology.orchestra.run.smithi142.stdout:(118/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-12T03:14:00.444 INFO:teuthology.orchestra.run.smithi142.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 604 kB/s | 172 kB 00:00 2024-09-12T03:14:00.477 INFO:teuthology.orchestra.run.smithi142.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-09-12T03:14:00.502 INFO:teuthology.orchestra.run.smithi142.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.8 MB/s | 220 kB 00:00 2024-09-12T03:14:00.527 INFO:teuthology.orchestra.run.smithi142.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-12T03:14:00.553 INFO:teuthology.orchestra.run.smithi142.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-12T03:14:00.578 INFO:teuthology.orchestra.run.smithi142.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-12T03:14:00.603 INFO:teuthology.orchestra.run.smithi142.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-12T03:14:00.617 INFO:teuthology.orchestra.run.smithi082.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 3.1 MB/s | 1.8 MB 00:00 2024-09-12T03:14:00.628 INFO:teuthology.orchestra.run.smithi142.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-12T03:14:00.642 INFO:teuthology.orchestra.run.smithi082.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 765 kB/s | 249 kB 00:00 2024-09-12T03:14:00.654 INFO:teuthology.orchestra.run.smithi142.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T03:14:00.679 INFO:teuthology.orchestra.run.smithi142.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-12T03:14:00.713 INFO:teuthology.orchestra.run.smithi142.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-12T03:14:00.738 INFO:teuthology.orchestra.run.smithi142.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:14:00.751 INFO:teuthology.orchestra.run.smithi082.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-12T03:14:00.772 INFO:teuthology.orchestra.run.smithi142.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-12T03:14:00.805 INFO:teuthology.orchestra.run.smithi142.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-12T03:14:00.831 INFO:teuthology.orchestra.run.smithi142.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-12T03:14:00.864 INFO:teuthology.orchestra.run.smithi142.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-12T03:14:00.889 INFO:teuthology.orchestra.run.smithi142.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:14:00.909 INFO:teuthology.orchestra.run.smithi082.stdout:(155/156): NetworkManager-1.49.90-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-12T03:14:00.914 INFO:teuthology.orchestra.run.smithi142.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-12T03:14:00.940 INFO:teuthology.orchestra.run.smithi142.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-12T03:14:00.965 INFO:teuthology.orchestra.run.smithi142.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:14:01.000 INFO:teuthology.orchestra.run.smithi142.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-12T03:14:01.024 INFO:teuthology.orchestra.run.smithi142.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-09-12T03:14:01.043 INFO:teuthology.orchestra.run.smithi082.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 2.4 MB 00:00 2024-09-12T03:14:01.048 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:14:01.049 INFO:teuthology.orchestra.run.smithi082.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-12T03:14:01.050 INFO:teuthology.orchestra.run.smithi142.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T03:14:01.075 INFO:teuthology.orchestra.run.smithi142.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:14:01.117 INFO:teuthology.orchestra.run.smithi142.stdout:(143/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-12T03:14:01.142 INFO:teuthology.orchestra.run.smithi142.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-12T03:14:01.159 INFO:teuthology.orchestra.run.smithi142.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-09-12T03:14:01.185 INFO:teuthology.orchestra.run.smithi142.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-12T03:14:01.210 INFO:teuthology.orchestra.run.smithi142.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-12T03:14:01.243 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-09-12T03:14:01.252 INFO:teuthology.orchestra.run.smithi142.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-12T03:14:01.442 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-09-12T03:14:01.442 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-09-12T03:14:01.477 INFO:teuthology.orchestra.run.smithi142.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T03:14:01.627 INFO:teuthology.orchestra.run.smithi142.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 3.2 MB 00:00 2024-09-12T03:14:01.919 INFO:teuthology.orchestra.run.smithi142.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 140 kB/s | 41 kB 00:00 2024-09-12T03:14:02.111 INFO:teuthology.orchestra.run.smithi142.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.7 MB/s | 2.3 MB 00:00 2024-09-12T03:14:02.220 INFO:teuthology.orchestra.run.smithi142.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 828 kB/s | 249 kB 00:00 2024-09-12T03:14:02.242 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-09-12T03:14:02.261 INFO:teuthology.orchestra.run.smithi142.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-12T03:14:02.303 INFO:teuthology.orchestra.run.smithi142.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 2.2 MB/s | 1.8 MB 00:00 2024-09-12T03:14:02.391 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-09-12T03:14:02.392 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-09-12T03:14:02.612 INFO:teuthology.orchestra.run.smithi142.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.1 MB/s | 2.4 MB 00:00 2024-09-12T03:14:02.618 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:14:02.618 INFO:teuthology.orchestra.run.smithi142.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-12T03:14:03.863 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-12T03:14:03.904 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-09-12T03:14:03.905 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-09-12T03:14:04.011 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-12T03:14:04.011 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-12T03:14:04.182 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-09-12T03:14:04.183 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-09-12T03:14:05.809 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-12T03:14:05.809 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-12T03:14:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-09-12T03:14:10.571 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:14:10.608 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-09-12T03:14:10.609 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:14:10.662 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:14:10.696 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:14:10.722 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:14:10.764 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:14:10.765 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:14:11.035 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:14:11.094 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:14:11.158 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:14:11.202 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:14:11.207 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:11.233 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:14:11.295 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:14:11.320 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:11.358 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:11.450 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:11.479 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:14:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:14:11.690 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:14:11.704 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:11.800 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:11.807 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:11.838 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:11.909 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:11.931 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:11.932 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:11.950 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:11.959 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:12.046 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-12T03:14:12.090 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:12.115 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:12.132 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:14:12.140 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:12.254 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:14:12.260 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:14:12.363 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:14:12.429 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:14:12.455 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:12.458 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:12.461 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:14:12.501 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:14:12.526 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:14:12.540 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:12.569 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:14:12.583 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:12.615 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:12.680 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:12.707 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:12.753 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:12.758 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:14:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:14:12.934 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:12.956 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:14:13.001 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:14:13.101 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:14:13.122 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:13.150 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:14:13.152 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:13.182 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:14:13.234 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:14:13.243 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:13.273 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:13.275 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:14:13.284 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:14:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:14:13.405 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:14:13.434 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:14:13.465 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:14:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:14:13.506 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:13.508 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:14:13.524 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:14:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:13.620 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:13.664 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:14:13.687 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:13.691 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:14:13.708 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:13.735 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:13.762 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:14:13.812 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:13.816 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:14:13.889 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:14:13.934 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:13.955 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:14:14.025 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:14:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:14:14.079 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:14:14.129 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:14:14.176 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:14:14.190 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:14:14.259 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:14:14.302 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:14.312 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:14:14.359 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:14:14.383 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:14.414 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:14:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:14:14.485 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:14:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:14:14.608 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:14:14.612 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:14:14.673 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:14:14.687 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:14:14.717 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:14:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:14:14.723 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:14:14.751 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:14:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:14:14.803 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:14:14.815 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:14:14.830 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:14:14.843 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:14:14.854 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:14:14.888 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:14:14.901 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:14:14.923 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:14:14.973 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:14:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:14:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:14:15.027 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:14:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:14:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:14:15.085 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:14:15.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:14:15.099 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:14:15.134 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:14:15.163 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:14:15.172 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:14:15.181 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:14:15.213 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:14:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:14:15.257 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:14:15.313 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:14:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:14:15.348 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:14:15.351 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:14:15.359 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:14:15.400 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:14:15.423 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:14:15.434 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:14:15.442 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:14:15.470 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:14:15.513 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:14:15.550 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:14:15.577 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:14:15.619 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:14:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:14:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:14:15.682 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:14:15.755 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:14:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:14:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:14:15.887 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:14:15.942 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:14:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:14:16.095 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:14:16.198 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:14:16.232 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:14:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:14:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:14:16.275 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:14:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:14:16.415 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:14:16.453 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:14:16.470 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:14:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:14:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:14:16.537 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:14:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:14:16.567 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:14:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:14:16.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:14:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:14:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:14:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:14:16.687 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:14:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:14:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:14:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:14:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:14:16.768 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:14:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:14:16.836 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:14:16.838 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:14:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:14:16.924 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:14:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:14:16.957 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:14:16.960 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:14:16.972 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:14:16.996 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:14:17.012 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:14:17.030 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:14:17.051 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:14:17.061 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:14:17.091 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:14:17.097 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:14:17.122 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:14:17.137 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:14:17.144 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:14:17.179 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:14:17.180 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:14:17.188 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:14:17.223 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:14:17.273 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:14:17.370 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:14:17.417 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:14:17.419 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:14:17.463 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:14:17.466 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:14:17.508 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:14:17.596 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:14:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:14:17.628 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:14:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:14:17.671 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:14:17.681 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:14:17.709 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:14:17.733 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:14:17.815 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:14:17.839 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:14:17.840 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:14:17.889 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:14:17.915 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:14:17.944 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:14:17.978 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:14:18.001 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:14:18.039 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:14:18.049 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:14:18.080 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:14:18.091 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:14:18.100 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:14:18.108 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:14:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:14:18.142 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:14:18.149 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:14:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:14:18.181 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:14:18.193 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:14:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:14:18.238 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:14:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:14:18.268 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:14:18.271 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:14:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:14:18.305 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:14:18.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:14:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:14:18.353 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:14:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:14:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:14:18.429 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:14:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:14:18.497 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:14:18.516 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:14:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:14:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:14:18.560 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:14:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:14:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:14:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:14:18.692 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:14:18.749 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:14:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:14:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:14:19.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:14:19.391 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:14:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:14:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:14:19.539 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:14:19.636 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:14:19.694 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:14:19.747 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:14:19.761 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:14:19.820 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:14:19.834 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:14:19.853 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:14:19.875 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:14:19.974 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:14:20.038 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:14:20.058 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:14:20.091 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:14:20.166 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:14:20.199 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:14:20.239 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:14:20.487 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:14:20.519 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:14:20.601 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:14:20.617 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:14:20.707 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:14:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:14:20.777 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:14:20.800 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:14:20.910 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:14:21.085 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:14:21.123 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:21.206 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:21.236 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:14:21.272 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:14:21.483 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:14:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:14:21.757 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:14:21.802 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:21.873 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:21.903 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:14:22.116 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:14:22.184 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:14:22.221 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:14:22.262 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:14:22.578 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:14:22.625 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:22.747 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:14:22.783 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:14:23.277 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:14:23.300 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:14:23.303 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:23.374 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:23.396 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:14:23.399 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:14:23.427 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:14:23.460 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:14:23.817 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:14:23.863 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:23.993 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:24.027 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:14:24.077 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:14:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:24.631 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:14:24.674 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:24.766 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:24.793 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:14:24.820 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:24.870 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:24.964 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:24.964 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:14:24.964 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:14:24.964 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:24.988 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:25.659 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:25.834 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:26.378 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:26.412 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:26.490 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:26.490 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:14:26.491 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:14:26.491 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:26.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:27.606 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:28.272 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:28.318 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:28.494 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:28.494 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:14:28.495 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:14:28.495 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-09-12T03:14:36.634 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:36.816 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:14:36.832 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:14:36.832 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:36.832 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:14:36.832 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-09-12T03:14:38.226 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-09-12T03:14:38.227 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-09-12T03:14:38.227 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:38.386 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:14:38.401 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:14:38.413 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:14:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:14:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:14:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:14:38.413 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:38.417 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:14:38.417 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:38.417 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:14:38.418 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:38.637 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:14:38.659 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:14:38.659 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:38.659 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:14:38.660 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:14:38.660 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:14:38.660 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:39.096 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:14:39.118 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:14:39.118 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:39.119 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:14:39.119 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:14:39.119 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:14:39.119 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:39.226 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:14:39.282 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:14:39.338 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:14:39.531 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:14:39.580 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:14:39.658 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:14:39.744 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:14:39.886 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:14:39.896 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:14:39.923 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:14:39.923 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:39.923 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:14:39.923 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:14:39.923 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:14:39.923 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:39.930 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:14:39.970 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:14:39.998 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:14:40.146 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:14:40.170 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:14:40.170 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:40.170 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:14:40.170 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:14:40.170 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:14:40.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:40.449 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:14:40.480 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:14:40.511 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:14:40.573 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:14:40.603 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:14:40.614 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:14:40.638 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:14:40.638 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:40.638 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:14:40.638 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:14:40.638 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:14:40.638 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:40.752 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:14:40.809 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:14:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:14:41.060 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:14:41.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:14:41.184 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:14:41.272 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:14:41.411 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:14:41.456 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:14:41.488 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:14:41.516 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:14:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:14:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:14:41.687 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-09-12T03:14:41.688 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:41.897 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:14:41.916 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:14:41.948 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:14:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:14:41.958 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:41.958 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:14:41.958 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:41.979 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:14:42.042 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:14:42.071 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:14:42.632 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:14:42.664 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:14:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:14:42.926 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:14:42.958 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:14:42.981 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:14:42.981 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:42.981 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:14:42.981 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:14:42.982 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:14:42.982 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:43.033 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:14:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:14:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:14:43.148 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:14:43.150 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:14:43.206 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:14:43.660 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:14:43.719 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:14:43.719 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:43.720 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:14:43.720 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:14:43.720 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:14:43.720 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:43.985 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:14:44.038 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:14:44.038 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.038 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:14:44.038 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:14:44.038 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:14:44.038 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:44.110 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:14:44.131 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:14:44.135 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:14:44.169 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:14:44.193 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:14:44.193 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.193 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:14:44.193 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:14:44.193 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:14:44.193 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:44.234 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:14:44.237 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:14:44.259 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:14:44.259 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.259 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:14:44.259 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:44.304 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:14:44.327 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:14:44.327 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.327 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:14:44.327 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:14:44.327 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:14:44.327 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:44.393 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:14:44.433 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:14:44.457 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:14:44.457 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.457 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:14:44.457 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:14:44.457 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:14:44.458 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:44.509 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:14:44.521 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:14:44.536 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:14:44.589 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:14:44.589 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.589 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:14:44.589 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:14:44.589 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:14:44.590 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:14:44.629 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:14:44.653 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:14:44.653 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:44.653 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:14:44.653 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:14:44.653 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:14:44.653 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:14:44.666 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:14:44.738 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:14:44.798 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:14:44.871 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:14:45.125 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:14:45.183 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:14:45.272 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:14:45.381 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:14:45.559 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:14:45.587 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:14:45.620 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:14:45.640 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:14:45.644 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:14:45.644 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:45.644 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:14:45.644 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:14:45.644 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:14:45.644 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:45.680 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:14:45.687 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:14:45.705 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:14:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:14:45.710 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:45.710 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:14:45.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:45.754 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:14:45.778 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:14:45.778 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:45.778 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:14:45.778 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:14:45.778 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:14:45.778 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:46.071 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:14:46.095 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:14:46.095 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:46.095 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:14:46.095 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:14:46.095 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:14:46.095 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:14:46.276 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:14:46.313 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:14:46.346 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:14:46.425 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:14:46.460 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:14:47.768 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:14:47.812 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:14:48.970 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:14:49.004 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:14:49.150 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:14:49.175 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:14:49.224 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:14:49.273 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:14:49.306 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:14:49.333 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:14:49.335 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:14:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:14:49.417 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:14:49.418 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:49.418 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:14:49.418 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:14:49.418 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:14:49.418 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:49.482 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:14:49.509 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:14:49.515 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:14:49.563 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:14:49.594 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:14:49.611 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:14:49.623 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:14:49.669 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:14:49.693 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:14:50.274 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:50.309 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:50.329 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:50.625 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:14:50.666 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:14:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:14:50.739 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:14:50.754 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:14:50.766 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:14:50.793 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:14:50.845 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:14:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:14:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:14:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:14:50.846 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:50.893 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:14:50.944 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:14:50.945 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:50.945 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:14:50.945 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:50.947 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:14:50.988 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:14:51.002 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:14:51.018 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:14:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:14:51.054 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:14:51.055 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:51.055 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:14:51.055 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:14:51.055 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:14:51.055 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:51.086 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:14:51.155 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:14:51.231 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:14:51.271 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:14:51.354 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:14:51.384 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:14:51.410 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:14:51.411 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:14:51.436 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:14:51.436 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:14:51.463 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:14:51.463 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:14:51.463 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:14:51.464 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:14:51.464 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:14:51.464 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:14:51.520 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:14:51.520 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:14:51.612 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:14:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:14:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:14:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:14:51.658 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:51.691 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:51.712 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:51.831 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:14:51.831 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:14:51.915 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:14:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:14:52.541 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:14:52.630 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:14:52.670 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:14:52.750 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:14:52.782 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:14:52.808 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:14:52.834 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:14:52.834 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:14:52.918 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:14:52.918 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:14:53.002 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:14:53.027 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:14:53.032 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:14:53.032 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:14:53.221 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:14:53.221 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:14:53.304 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:14:53.368 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:14:56.725 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:14:56.776 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:14:56.820 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:14:56.856 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:14:56.892 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:14:57.143 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:14:57.197 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:14:57.246 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:14:57.292 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:14:57.360 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:14:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:14:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:14:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:14:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:14:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:14:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:14:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:14:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:14:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:14:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:14:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:14:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:14:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:14:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:14:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:14:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:14:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:14:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:14:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:14:58.144 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:58.191 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:58.234 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:14:58.900 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:14:58.901 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:14:58.902 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:14:58.918 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:14:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:14:58.920 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:14:58.921 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:14:58.922 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:14:58.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:14:58.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:14:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:14:58.926 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:14:58.927 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:14:58.928 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:14:58.929 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:14:59.610 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:14:59.722 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:14:59.779 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:14:59.894 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:14:59.933 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:14:59.958 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:15:00.003 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:15:00.003 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:15:00.127 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.128 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.129 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.130 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:15:00.131 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.132 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:00.133 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.134 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:15:00.135 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:15:00.136 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:15:00.137 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:15:00.138 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:15:00.139 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:15:00.140 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:15:00.140 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:15:00.140 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:15:00.140 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:15:00.140 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-09-12T03:15:00.236 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:15:00.260 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:15:00.277 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:00.277 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:00.521 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:00.521 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:15:00.562 DEBUG:teuthology.parallel:result is None 2024-09-12T03:15:00.631 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:15:00.711 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:00.927 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:00.927 INFO:teuthology.orchestra.run.smithi050.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-12T03:15:00.927 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:15:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:15:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:15:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:15:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:15:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:15:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:15:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:15:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:15:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:15:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:15:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:15:01.506 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-12T03:15:01.908 DEBUG:teuthology.parallel:result is None 2024-09-12T03:15:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:15:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:15:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:15:06.714 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:15:06.715 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:15:06.716 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:15:06.728 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:15:06.729 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:15:06.730 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:15:06.731 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:15:06.732 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:15:06.734 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:15:06.735 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:15:06.736 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:15:06.737 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:15:06.738 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:15:06.738 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:15:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:15:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:15:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:15:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:15:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:15:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:15:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:15:08.888 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-09-12T03:15:09.515 DEBUG:teuthology.parallel:result is None 2024-09-12T03:15:09.515 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:09.655 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:15:09.701 INFO:teuthology.orchestra.run.smithi050.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:09.702 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:09.702 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:15:09.704 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:09.841 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:15:09.868 INFO:teuthology.orchestra.run.smithi082.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:09.868 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:09.868 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:15:09.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:10.008 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:15:10.035 INFO:teuthology.orchestra.run.smithi142.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:10.035 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:10.035 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:15:10.037 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T03:15:10.037 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:15:10.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:15:10.095 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:15:10.095 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:15:10.127 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:15:10.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:15:10.159 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T03:15:10.159 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:15:10.159 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:15:10.217 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:15:10.320 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:15:10.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:15:10.350 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:15:10.419 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:15:10.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:15:10.450 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:15:10.518 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T03:15:10.518 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:15:10.518 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:15:10.574 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:15:10.672 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:15:10.672 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:15:10.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:15:10.770 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:15:10.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:15:10.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:15:10.868 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T03:15:10.869 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:15:10.869 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:15:10.927 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:15:11.026 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:15:11.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:15:11.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:15:11.124 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:15:11.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:15:11.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:15:11.223 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T03:15:11.311 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2}} 2024-09-12T03:15:11.311 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:11.312 INFO:tasks.cephadm:Cluster fsid is 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:15:11.312 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T03:15:11.312 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.50', 'mon.b': '172.21.15.82', 'mon.c': '172.21.15.142'} 2024-09-12T03:15:11.312 INFO:tasks.cephadm:First mon is mon.a on smithi050 2024-09-12T03:15:11.312 INFO:tasks.cephadm:First mgr is x 2024-09-12T03:15:11.312 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T03:15:11.313 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-09-12T03:15:11.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-09-12T03:15:11.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-09-12T03:15:11.437 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T03:15:11.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:11.579 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T03:15:11.721 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:11.722 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:15:11.723 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:15:11.723 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.134 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:15 /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.135 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.536 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:15 /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.536 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.921 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:15 /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.921 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.960 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:12.979 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:15:13.018 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T03:15:13.018 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:15:13.022 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:15:13.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:15:14.578 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:15:14.617 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:15:15.169 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:15:48.176 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:15:48.176 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:15:48.176 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:15:48.177 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-09-12T03:15:48.177 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:15:48.177 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:15:48.177 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:15:51.374 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-09-12T03:15:51.374 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:15:51.374 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:15:51.374 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-09-12T03:15:51.375 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:15:51.375 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-09-12T03:15:51.375 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-09-12T03:16:07.222 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T03:16:07.222 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:16:07.222 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:16:07.223 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-09-12T03:16:07.223 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:16:07.223 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-12T03:16:07.223 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T03:16:07.248 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-09-12T03:16:07.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-09-12T03:16:07.347 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-09-12T03:16:07.375 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-09-12T03:16:07.434 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-09-12T03:16:07.463 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-09-12T03:16:07.488 INFO:tasks.cephadm:Writing seed config... 2024-09-12T03:16:07.489 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T03:16:07.489 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T03:16:07.489 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T03:16:07.490 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T03:16:07.491 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T03:16:07.492 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-12T03:16:07.493 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:16:07.494 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T03:16:07.530 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 389f22da-70b5-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T03:16:07.530 DEBUG:teuthology.orchestra.run.smithi050:mon.a> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T03:16:07.575 DEBUG:teuthology.orchestra.run.smithi050:mgr.x> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T03:16:07.620 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T03:16:07.620 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.50 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:16:07.886 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:16:07.887 INFO:teuthology.orchestra.run.smithi050.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '389f22da-70b5-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.50', '--skip-admin-label'] 2024-09-12T03:16:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T03:16:07.887 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying podman|docker is present... 2024-09-12T03:16:07.932 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:16:07.932 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying lvm2 is present... 2024-09-12T03:16:07.932 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying time synchronization is in place... 2024-09-12T03:16:07.946 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:16:07.946 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:16:07.959 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:16:07.959 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-09-12T03:16:07.973 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout enabled 2024-09-12T03:16:07.985 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout active 2024-09-12T03:16:07.986 INFO:teuthology.orchestra.run.smithi050.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:16:07.986 INFO:teuthology.orchestra.run.smithi050.stdout:Repeating the final host check... 2024-09-12T03:16:08.029 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:16:08.030 INFO:teuthology.orchestra.run.smithi050.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T03:16:08.030 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl is present 2024-09-12T03:16:08.030 INFO:teuthology.orchestra.run.smithi050.stdout:lvcreate is present 2024-09-12T03:16:08.042 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:16:08.042 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:16:08.054 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:16:08.054 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-09-12T03:16:08.068 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout enabled 2024-09-12T03:16:08.081 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout active 2024-09-12T03:16:08.081 INFO:teuthology.orchestra.run.smithi050.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:16:08.081 INFO:teuthology.orchestra.run.smithi050.stdout:Host looks OK 2024-09-12T03:16:08.081 INFO:teuthology.orchestra.run.smithi050.stdout:Cluster fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:08.081 INFO:teuthology.orchestra.run.smithi050.stdout:Acquiring lock 139820215673712 on /run/cephadm/389f22da-70b5-11ef-bcea-c7b262605968.lock 2024-09-12T03:16:08.081 INFO:teuthology.orchestra.run.smithi050.stdout:Lock 139820215673712 acquired on /run/cephadm/389f22da-70b5-11ef-bcea-c7b262605968.lock 2024-09-12T03:16:08.082 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying IP 172.21.15.50 port 3300 ... 2024-09-12T03:16:08.082 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying IP 172.21.15.50 port 6789 ... 2024-09-12T03:16:08.082 INFO:teuthology.orchestra.run.smithi050.stdout:Base mon IP(s) is [172.21.15.50:3300, 172.21.15.50:6789], mon addrv is [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-09-12T03:16:08.089 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.50 metric 100 2024-09-12T03:16:08.089 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.50 metric 100 2024-09-12T03:16:08.095 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T03:16:08.095 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T03:16:08.095 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1784sec hoplimit 64 pref medium 2024-09-12T03:16:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T03:16:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T03:16:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:16:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T03:16:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-12T03:16:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:16:08.102 INFO:teuthology.orchestra.run.smithi050.stdout:Mon IP `172.21.15.50` is in CIDR network `172.21.0.0/20` 2024-09-12T03:16:08.102 INFO:teuthology.orchestra.run.smithi050.stdout:Mon IP `172.21.15.50` is in CIDR network `172.21.0.0/20` 2024-09-12T03:16:08.102 INFO:teuthology.orchestra.run.smithi050.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T03:16:08.103 INFO:teuthology.orchestra.run.smithi050.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T03:16:08.104 INFO:teuthology.orchestra.run.smithi050.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:16:08.640 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:16:08.640 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:16:08.641 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T03:16:08.641 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T03:16:08.641 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T03:16:08.641 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:16:08.641 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T03:16:09.609 INFO:teuthology.orchestra.run.smithi050.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:16:09.610 INFO:teuthology.orchestra.run.smithi050.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:16:09.610 INFO:teuthology.orchestra.run.smithi050.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T03:16:10.682 INFO:teuthology.orchestra.run.smithi050.stdout:stat: stdout 167 167 2024-09-12T03:16:10.682 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial keys... 2024-09-12T03:16:11.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQD7XOJm90p8BBAAAJUW9OAQyitKN0DWwHJh/Q== 2024-09-12T03:16:12.737 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQD8XOJm7OhcBhAAMe8Hb+xqb33kJF0mw3gnKA== 2024-09-12T03:16:13.809 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQD9XOJmJwTADRAAmDd2DbF05z6dXzQ09Dg/lw== 2024-09-12T03:16:13.811 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial monmap... 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool for a [v2:172.21.15.50:3300,v1:172.21.15.50:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:setting min_mon_release = quincy 2024-09-12T03:16:14.783 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: set fsid to 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:14.784 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:16:14.784 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:16:14.784 INFO:teuthology.orchestra.run.smithi050.stdout:Creating mon... 2024-09-12T03:16:15.788 INFO:teuthology.orchestra.run.smithi050.stdout:create mon.a on 2024-09-12T03:16:16.107 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T03:16:16.333 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T03:16:16.602 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-389f22da-70b5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-389f22da-70b5-11ef-bcea-c7b262605968.target. 2024-09-12T03:16:16.603 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-389f22da-70b5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-389f22da-70b5-11ef-bcea-c7b262605968.target. 2024-09-12T03:16:16.907 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:16:16.908 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to reset failed state of unit ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T03:16:17.175 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-389f22da-70b5-11ef-bcea-c7b262605968.target.wants/ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-389f22da-70b5-11ef-bcea-c7b262605968@.service. 2024-09-12T03:16:17.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 podman[29990]: 2024-09-12 03:16:17.44069455 +0000 UTC m=+0.035182037 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:16:17.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 podman[29990]: 2024-09-12 03:16:17.779978583 +0000 UTC m=+0.374466070 container init 42e2e320a0fb4572c15f8f0a4c76ba2c61337bc3caeec0711af9ec9730754b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:16:17.891 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:16:17.891 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-09-12T03:16:17.905 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:16:17.905 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-09-12T03:16:17.905 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-09-12T03:16:17.905 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:16:17.906 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mon to start... 2024-09-12T03:16:17.906 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mon... 2024-09-12T03:16:18.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 podman[29990]: 2024-09-12 03:16:17.789630683 +0000 UTC m=+0.384118170 container start 42e2e320a0fb4572c15f8f0a4c76ba2c61337bc3caeec0711af9ec9730754b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:16:18.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 ceph-mon[30012]: mkfs 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:18.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 ceph-mon[30012]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:16:18.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 bash[29990]: 42e2e320a0fb4572c15f8f0a4c76ba2c61337bc3caeec0711af9ec9730754b20 2024-09-12T03:16:18.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:17 smithi050 systemd[1]: Started Ceph mon.a for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: monmap epoch 1 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: last_changed 2024-09-12T03:16:14.215514+0000 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: min_mon_release 19 (squid) 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: election_strategy: 1 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: fsmap 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: mgrmap e1: no daemons active 2024-09-12T03:16:19.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:18 smithi050 ceph-mon[30012]: from='client.? 172.21.15.50:0/672730081' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T03:16:19.099 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout id: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout services: 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.623086s) 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout data: 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T03:16:19.100 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.392+0000 7f594b200640 1 Processor -- start 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.392+0000 7f594b200640 1 -- start start 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.392+0000 7f594b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c07b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.392+0000 7f594b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f594c07b7a0 con 0x7f594c07cd70 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.393+0000 7f5949e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c07b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:19.101 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.393+0000 7f5949e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c07b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32852/0 (socket says 172.21.15.50:32852) 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.393+0000 7f5949e00640 1 -- 172.21.15.50:0/1659034219 learned_addr learned my addr 172.21.15.50:0/1659034219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.393+0000 7f5949e00640 1 -- 172.21.15.50:0/1659034219 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c07b920 con 0x7f594c07cd70 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.396+0000 7f5949e00640 1 --2- 172.21.15.50:0/1659034219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c07b1d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f593c009820 tx=0x7f593c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a2f7c2e7e7e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.396+0000 7f5948a00640 1 -- 172.21.15.50:0/1659034219 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f593c037070 con 0x7f594c07cd70 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.396+0000 7f5948a00640 1 -- 172.21.15.50:0/1659034219 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c032440 con 0x7f594c07cd70 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.396+0000 7f5948a00640 1 -- 172.21.15.50:0/1659034219 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f593c032740 con 0x7f594c07cd70 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 -- 172.21.15.50:0/1659034219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 msgr2=0x7f594c07b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 --2- 172.21.15.50:0/1659034219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c07b1d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f593c009820 tx=0x7f593c02a270 comp rx=0 tx=0).stop 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 -- 172.21.15.50:0/1659034219 shutdown_connections 2024-09-12T03:16:19.105 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 --2- 172.21.15.50:0/1659034219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c07b1d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 -- 172.21.15.50:0/1659034219 >> 172.21.15.50:0/1659034219 conn(0x7f594c102b90 msgr2=0x7f594c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 -- 172.21.15.50:0/1659034219 shutdown_connections 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.397+0000 7f594b200640 1 -- 172.21.15.50:0/1659034219 wait complete. 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.398+0000 7f594b200640 1 Processor -- start 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.398+0000 7f594b200640 1 -- start start 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.399+0000 7f594b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.399+0000 7f594b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f594c07bec0 con 0x7f594c07cd70 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.399+0000 7f5949e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.399+0000 7f5949e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:32856/0 (socket says 172.21.15.50:32856) 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.399+0000 7f5949e00640 1 -- 172.21.15.50:0/672730081 learned_addr learned my addr 172.21.15.50:0/672730081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.399+0000 7f5949e00640 1 -- 172.21.15.50:0/672730081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c19d280 con 0x7f594c07cd70 2024-09-12T03:16:19.106 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.400+0000 7f5949e00640 1 --2- 172.21.15.50:0/672730081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c19cd40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f593c0090b0 tx=0x7f593c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.400+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f593c041070 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.400+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c03c7a0 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.400+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f593c037040 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.400+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c19d510 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.400+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c1a0200 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.401+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f593c044070 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.401+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f593c03d3f0 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.402+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594c07b250 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.405+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f593c046450 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.469+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f594c02d8a0 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.470+0000 7f5942a00640 1 -- 172.21.15.50:0/672730081 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f593c046630 con 0x7f594c07cd70 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 msgr2=0x7f594c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:19.107 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 --2- 172.21.15.50:0/672730081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c19cd40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f593c0090b0 tx=0x7f593c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 shutdown_connections 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 --2- 172.21.15.50:0/672730081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f594c07cd70 0x7f594c19cd40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 >> 172.21.15.50:0/672730081 conn(0x7f594c102b90 msgr2=0x7f594c103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 shutdown_connections 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:18.471+0000 7f594b200640 1 -- 172.21.15.50:0/672730081 wait complete. 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:mon is available 2024-09-12T03:16:19.108 INFO:teuthology.orchestra.run.smithi050.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T03:16:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:19 smithi050 ceph-mon[30012]: from='client.? 172.21.15.50:0/699113427' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:16:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:19 smithi050 ceph-mon[30012]: from='client.? 172.21.15.50:0/699113427' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T03:16:20.471 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout fsid = 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:20.472 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.764+0000 7f1ef2200640 1 Processor -- start 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.765+0000 7f1ef2200640 1 -- start start 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.765+0000 7f1ef2200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.765+0000 7f1ef2200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec10a1e0 con 0x7f1eec109810 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.766+0000 7f1ef0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.766+0000 7f1ef0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36932/0 (socket says 172.21.15.50:36932) 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.766+0000 7f1ef0e00640 1 -- 172.21.15.50:0/1065883000 learned_addr learned my addr 172.21.15.50:0/1065883000 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:20.473 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.766+0000 7f1ef0e00640 1 -- 172.21.15.50:0/1065883000 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec10a730 con 0x7f1eec109810 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.767+0000 7f1ef0e00640 1 --2- 172.21.15.50:0/1065883000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec109c10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009820 tx=0x7f1ee002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c726f1e27f7b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1eeb400640 1 -- 172.21.15.50:0/1065883000 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0037070 con 0x7f1eec109810 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1eeb400640 1 -- 172.21.15.50:0/1065883000 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee0032440 con 0x7f1eec109810 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1eeb400640 1 -- 172.21.15.50:0/1065883000 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0032740 con 0x7f1eec109810 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1ef2200640 1 -- 172.21.15.50:0/1065883000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 msgr2=0x7f1eec109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1ef2200640 1 --2- 172.21.15.50:0/1065883000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec109c10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009820 tx=0x7f1ee002a270 comp rx=0 tx=0).stop 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1ef2200640 1 -- 172.21.15.50:0/1065883000 shutdown_connections 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1ef2200640 1 --2- 172.21.15.50:0/1065883000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec109c10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.768+0000 7f1ef2200640 1 -- 172.21.15.50:0/1065883000 >> 172.21.15.50:0/1065883000 conn(0x7f1eec07cb00 msgr2=0x7f1eec07cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.769+0000 7f1ef2200640 1 -- 172.21.15.50:0/1065883000 shutdown_connections 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.769+0000 7f1ef2200640 1 -- 172.21.15.50:0/1065883000 wait complete. 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef2200640 1 Processor -- start 2024-09-12T03:16:20.474 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef2200640 1 -- start start 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef2200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef2200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec10ab50 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36940/0 (socket says 172.21.15.50:36940) 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.770+0000 7f1ef0e00640 1 -- 172.21.15.50:0/699113427 learned_addr learned my addr 172.21.15.50:0/699113427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.771+0000 7f1ef0e00640 1 -- 172.21.15.50:0/699113427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec19d2a0 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.771+0000 7f1ef0e00640 1 --2- 172.21.15.50:0/699113427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec19cd60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00090b0 tx=0x7f1ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.771+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0041070 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.771+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee003c7a0 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.771+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ee0037040 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.771+0000 7f1ef2200640 1 -- 172.21.15.50:0/699113427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec19d530 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.772+0000 7f1ef2200640 1 -- 172.21.15.50:0/699113427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec1a0220 con 0x7f1eec109810 2024-09-12T03:16:20.475 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.772+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ee0044070 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.772+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ee003d730 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.773+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eec109c90 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.775+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1ee003cbe0 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.838+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1eec19da10 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.842+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f1eec19da10 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.843+0000 7f1ee9600640 1 -- 172.21.15.50:0/699113427 <== mon.0 v2:172.21.15.50:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1ee0032440 con 0x7f1eec109810 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 msgr2=0x7f1eec19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 --2- 172.21.15.50:0/699113427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec19cd60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00090b0 tx=0x7f1ee00046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 shutdown_connections 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 --2- 172.21.15.50:0/699113427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eec109810 0x7f1eec19cd60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 >> 172.21.15.50:0/699113427 conn(0x7f1eec07cb00 msgr2=0x7f1eec190fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:20.476 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 shutdown_connections 2024-09-12T03:16:20.477 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:19.844+0000 7f1edea00640 1 -- 172.21.15.50:0/699113427 wait complete. 2024-09-12T03:16:20.477 INFO:teuthology.orchestra.run.smithi050.stdout:Generating new minimal ceph.conf... 2024-09-12T03:16:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:21 smithi050 ceph-mon[30012]: from='client.? 172.21.15.50:0/2250048340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.958+0000 7f87b8000640 1 Processor -- start 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.958+0000 7f87b8000640 1 -- start start 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.959+0000 7f87b8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.959+0000 7f87b8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87b007c870 con 0x7f87b007de40 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.959+0000 7f87b5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.959+0000 7f87b5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36956/0 (socket says 172.21.15.50:36956) 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.959+0000 7f87b5600640 1 -- 172.21.15.50:0/3010423317 learned_addr learned my addr 172.21.15.50:0/3010423317 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.959+0000 7f87b5600640 1 -- 172.21.15.50:0/3010423317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b007c9f0 con 0x7f87b007de40 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.960+0000 7f87b5600640 1 --2- 172.21.15.50:0/3010423317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b007c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009820 tx=0x7f87a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6543c19cf1aafe80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:21.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87afe00640 1 -- 172.21.15.50:0/3010423317 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87a4037070 con 0x7f87b007de40 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87afe00640 1 -- 172.21.15.50:0/3010423317 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87a4032900 con 0x7f87b007de40 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87afe00640 1 -- 172.21.15.50:0/3010423317 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87a4030850 con 0x7f87b007de40 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87b8000640 1 -- 172.21.15.50:0/3010423317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 msgr2=0x7f87b007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87b8000640 1 --2- 172.21.15.50:0/3010423317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b007c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009820 tx=0x7f87a402a270 comp rx=0 tx=0).stop 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87b8000640 1 -- 172.21.15.50:0/3010423317 shutdown_connections 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87b8000640 1 --2- 172.21.15.50:0/3010423317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b007c2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87b8000640 1 -- 172.21.15.50:0/3010423317 >> 172.21.15.50:0/3010423317 conn(0x7f87b0102cc0 msgr2=0x7f87b0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.961+0000 7f87b8000640 1 -- 172.21.15.50:0/3010423317 shutdown_connections 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.962+0000 7f87b8000640 1 -- 172.21.15.50:0/3010423317 wait complete. 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.962+0000 7f87b8000640 1 Processor -- start 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.962+0000 7f87b8000640 1 -- start start 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.963+0000 7f87b8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:21.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.963+0000 7f87b8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87b010dba0 con 0x7f87b007de40 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.963+0000 7f87b5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.963+0000 7f87b5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36968/0 (socket says 172.21.15.50:36968) 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.963+0000 7f87b5600640 1 -- 172.21.15.50:0/2250048340 learned_addr learned my addr 172.21.15.50:0/2250048340 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.963+0000 7f87b5600640 1 -- 172.21.15.50:0/2250048340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b019dc80 con 0x7f87b007de40 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87b5600640 1 --2- 172.21.15.50:0/2250048340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b019d740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f87a40097f0 tx=0x7f87a4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87a4037040 con 0x7f87b007de40 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87a4040070 con 0x7f87b007de40 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b019df10 con 0x7f87b007de40 2024-09-12T03:16:21.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87a403a390 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b01a0c00 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87a403a5e0 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.964+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87a4045950 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.965+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b007c320 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:20.968+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f87a4045460 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.031+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f87b01a0e20 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.031+0000 7f87ae000640 1 -- 172.21.15.50:0/2250048340 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f87a403b430 con 0x7f87b007de40 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 msgr2=0x7f87b019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 --2- 172.21.15.50:0/2250048340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b019d740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f87a40097f0 tx=0x7f87a4030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:21.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 shutdown_connections 2024-09-12T03:16:21.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 --2- 172.21.15.50:0/2250048340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87b007de40 0x7f87b019d740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 >> 172.21.15.50:0/2250048340 conn(0x7f87b0102cc0 msgr2=0x7f87b0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:21.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 shutdown_connections 2024-09-12T03:16:21.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:21.032+0000 7f87b8000640 1 -- 172.21.15.50:0/2250048340 wait complete. 2024-09-12T03:16:21.622 INFO:teuthology.orchestra.run.smithi050.stdout:Restarting the monitor... 2024-09-12T03:16:21.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:21 smithi050 systemd[1]: Stopping Ceph mon.a for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:16:22.267 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a[30006]: 2024-09-12T03:16:21.972+0000 7fdc88e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:16:22.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a[30006]: 2024-09-12T03:16:21.972+0000 7fdc88e00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T03:16:22.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:21 smithi050 podman[30260]: 2024-09-12 03:16:21.99467184 +0000 UTC m=+0.129173012 container died 42e2e320a0fb4572c15f8f0a4c76ba2c61337bc3caeec0711af9ec9730754b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T03:16:22.537 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 podman[30260]: 2024-09-12 03:16:22.378062707 +0000 UTC m=+0.512563877 container cleanup 42e2e320a0fb4572c15f8f0a4c76ba2c61337bc3caeec0711af9ec9730754b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:16:22.537 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 bash[30260]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:16:22.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 podman[30272]: 2024-09-12 03:16:22.537428058 +0000 UTC m=+0.533847281 container remove 42e2e320a0fb4572c15f8f0a4c76ba2c61337bc3caeec0711af9ec9730754b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:16:23.123 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T03:16:23.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 systemd[1]: Stopped Ceph mon.a for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:16:23.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.227s CPU time. 2024-09-12T03:16:23.124 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:22 smithi050 systemd[1]: Starting Ceph mon.a for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:16:23.381 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 podman[30360]: 2024-09-12 03:16:23.028417857 +0000 UTC m=+0.027516699 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:16:23.382 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 podman[30360]: 2024-09-12 03:16:23.147001509 +0000 UTC m=+0.146100350 container create 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T03:16:23.382 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 podman[30360]: 2024-09-12 03:16:23.341224936 +0000 UTC m=+0.340323778 container init 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T03:16:23.382 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 podman[30360]: 2024-09-12 03:16:23.347571247 +0000 UTC m=+0.346670090 container start 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T03:16:23.441 INFO:teuthology.orchestra.run.smithi050.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T03:16:23.654 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: pidfile_write: ignore empty --pid-file 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: load: jerasure load: lrc 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Git sha 0 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: DB SUMMARY 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: DB Session ID: 61B8CQEAIVFYQTZNP3DS 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T03:16:23.655 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104676 ; 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.env: 0x56099b8d9940 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.info_log: 0x56099c41f6a0 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.statistics: (nil) 2024-09-12T03:16:23.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.use_fsync: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.db_log_dir: 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.wal_dir: 2024-09-12T03:16:23.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:16:23.659 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.write_buffer_manager: 0x56099c42edc0 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.unordered_write: 0 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:16:23.660 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.row_cache: None 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.wal_filter: None 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.wal_compression: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:16:23.661 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:16:23.662 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_open_files: -1 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Compression algorithms supported: 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kZSTD supported: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:16:23.663 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.merge_operator: 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_filter: None 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:16:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56099c41f300) 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_type: 0 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: data_block_index_type: 0 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_shortening: 1 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: checksum: 4 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: no_block_cache: 0 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache: 0x56099c4131f0 2024-09-12T03:16:23.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_options: 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: capacity : 536870912 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: num_shard_bits : 4 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: strict_capacity_limit : 0 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_compressed: (nil) 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: persistent_cache: (nil) 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_size: 4096 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_size_deviation: 10 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_restart_interval: 16 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_block_restart_interval: 1 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: metadata_block_size: 4096 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: partition_filters: 0 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: use_delta_encoding: 1 2024-09-12T03:16:23.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: filter_policy: bloomfilter 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: whole_key_filtering: 1 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: verify_compression: 0 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: format_version: 5 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: enable_index_compression: 1 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_align: 0 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: max_auto_readahead_size: 262144 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: prepopulate_block_cache: 0 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression: NoCompression 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:16:23.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.num_levels: 7 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:16:23.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:16:23.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:16:23.670 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:16:23.671 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.ttl: 2592000 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:16:23.672 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a154f1f3-72c2-479c-8354-d942f49cf0c4 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110983390319, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T03:16:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110983392159, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99831, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97713, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14159, "raw_average_key_size": 49, "raw_value_size": 90022, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726110983, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a154f1f3-72c2-479c-8354-d942f49cf0c4", "db_session_id": "61B8CQEAIVFYQTZNP3DS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110983392415, "job": 1, "event": "recovery_finished"} 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56099c442e00 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: DB pointer 0x56099c53e000 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** DB Stats ** 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:16:23.674 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** Compaction Stats [default] ** 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: L0 2/0 99.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: Sum 2/0 99.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** Compaction Stats [default] ** 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 80.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-09-12T03:16:23.675 INFO:journalctl@ceph.mon.a.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative compaction: 0.00 GB write, 15.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval compaction: 0.00 GB write, 15.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Block cache BinnedLRUCache@0x56099c4131f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:16:23.676 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: starting mon.a rank 0 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???) e1 preinit fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).mds e1 new map 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).mds e1 print_map 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: e1 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: btime 2024-09-12T03:16:17:849478+0000 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: legacy client fscid: -1 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout: No filesystems configured 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:16:23.677 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 bash[30360]: 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 systemd[1]: Started Ceph mon.a for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: monmap epoch 1 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: last_changed 2024-09-12T03:16:14.215514+0000 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: min_mon_release 19 (squid) 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: election_strategy: 1 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: fsmap 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:16:23.678 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:23 smithi050 ceph-mon[30378]: mgrmap e1: no daemons active 2024-09-12T03:16:24.852 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.880+0000 7fc3ecc00640 1 Processor -- start 2024-09-12T03:16:24.856 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.880+0000 7fc3ecc00640 1 -- start start 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.880+0000 7fc3ecc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e8109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.880+0000 7fc3ecc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e810a1e0 con 0x7fc3e8109810 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.881+0000 7fc3e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e8109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.881+0000 7fc3e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e8109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36982/0 (socket says 172.21.15.50:36982) 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.881+0000 7fc3e5e00640 1 -- 172.21.15.50:0/1345729219 learned_addr learned my addr 172.21.15.50:0/1345729219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.881+0000 7fc3e5e00640 1 -- 172.21.15.50:0/1345729219 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e810a950 con 0x7fc3e8109810 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.884+0000 7fc3e5e00640 1 --2- 172.21.15.50:0/1345729219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e8109c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009820 tx=0x7fc3d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a0623a46a162d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.884+0000 7fc3e5400640 1 -- 172.21.15.50:0/1345729219 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8037070 con 0x7fc3e8109810 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.884+0000 7fc3e5400640 1 -- 172.21.15.50:0/1345729219 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3d8032900 con 0x7fc3e8109810 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.884+0000 7fc3e5400640 1 -- 172.21.15.50:0/1345729219 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8030850 con 0x7fc3e8109810 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 -- 172.21.15.50:0/1345729219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 msgr2=0x7fc3e8109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.857 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 --2- 172.21.15.50:0/1345729219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e8109c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009820 tx=0x7fc3d802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 -- 172.21.15.50:0/1345729219 shutdown_connections 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 --2- 172.21.15.50:0/1345729219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e8109c10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 -- 172.21.15.50:0/1345729219 >> 172.21.15.50:0/1345729219 conn(0x7fc3e807ccf0 msgr2=0x7fc3e807d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 -- 172.21.15.50:0/1345729219 shutdown_connections 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.885+0000 7fc3ecc00640 1 -- 172.21.15.50:0/1345729219 wait complete. 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.886+0000 7fc3ecc00640 1 Processor -- start 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.886+0000 7fc3ecc00640 1 -- start start 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.886+0000 7fc3ecc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.886+0000 7fc3ecc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e810dd00 con 0x7fc3e8109810 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.887+0000 7fc3e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.887+0000 7fc3e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36994/0 (socket says 172.21.15.50:36994) 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.887+0000 7fc3e5e00640 1 -- 172.21.15.50:0/3861405150 learned_addr learned my addr 172.21.15.50:0/3861405150 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:24.858 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.887+0000 7fc3e5e00640 1 -- 172.21.15.50:0/3861405150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819dd10 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.887+0000 7fc3e5e00640 1 --2- 172.21.15.50:0/3861405150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e819d7d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009940 tx=0x7fc3d80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.887+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8042070 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.888+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3d8032be0 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.888+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819dfa0 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.888+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d8037040 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.888+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a0c90 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.888+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3d8045070 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.888+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3d8047a80 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.889+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e8109c10 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.892+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc3d8032d90 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.954+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fc3e81a0e60 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.956+0000 7fc3dea00640 1 -- 172.21.15.50:0/3861405150 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fc3d803d430 con 0x7fc3e8109810 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.957+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 msgr2=0x7fc3e819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.859 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.957+0000 7fc3ecc00640 1 --2- 172.21.15.50:0/3861405150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e819d7d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009940 tx=0x7fc3d80326e0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.860 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.957+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 shutdown_connections 2024-09-12T03:16:24.860 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.957+0000 7fc3ecc00640 1 --2- 172.21.15.50:0/3861405150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3e8109810 0x7fc3e819d7d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.860 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.957+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 >> 172.21.15.50:0/3861405150 conn(0x7fc3e807ccf0 msgr2=0x7fc3e81074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.860 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.958+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 shutdown_connections 2024-09-12T03:16:24.860 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:23.958+0000 7fc3ecc00640 1 -- 172.21.15.50:0/3861405150 wait complete. 2024-09-12T03:16:24.860 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T03:16:24.879 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:16:24.880 INFO:teuthology.orchestra.run.smithi050.stdout:Creating mgr... 2024-09-12T03:16:24.880 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T03:16:24.881 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T03:16:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:24 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3861405150' entity='client.admin' 2024-09-12T03:16:25.230 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:16:25.231 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to reset failed state of unit ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T03:16:25.466 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-389f22da-70b5-11ef-bcea-c7b262605968.target.wants/ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-389f22da-70b5-11ef-bcea-c7b262605968@.service. 2024-09-12T03:16:25.664 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:25 smithi050 systemd[1]: Starting Ceph mgr.x for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:16:26.094 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:25 smithi050 podman[30589]: 2024-09-12 03:16:25.79148675 +0000 UTC m=+0.112741066 container create d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T03:16:26.094 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:25 smithi050 podman[30589]: 2024-09-12 03:16:25.705827001 +0000 UTC m=+0.027081341 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:16:26.094 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 podman[30589]: 2024-09-12 03:16:26.004521449 +0000 UTC m=+0.325775765 container init d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T03:16:26.094 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 podman[30589]: 2024-09-12 03:16:26.010530034 +0000 UTC m=+0.331784350 container start d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:16:26.094 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 bash[30589]: d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 2024-09-12T03:16:26.123 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:16:26.124 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-09-12T03:16:26.139 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:16:26.139 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-09-12T03:16:26.139 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-09-12T03:16:26.139 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:16:26.157 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:16:26.157 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-09-12T03:16:26.171 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:16:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-09-12T03:16:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-09-12T03:16:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T03:16:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr to start... 2024-09-12T03:16:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr... 2024-09-12T03:16:26.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 systemd[1]: Started Ceph mgr.x for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:16:26.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:26.189+0000 7f4db5632040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:16:26.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:26.263+0000 7f4db5632040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:26.692 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:26.463+0000 7f4db5632040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:16:26.692 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:26.546+0000 7f4db5632040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:16:27.092 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/918555117' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:16:27.092 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:26.733+0000 7f4db5632040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:27.092 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:26.909+0000 7f4db5632040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:16:27.350 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:27 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:27.322+0000 7f4db5632040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:16:27.368 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:16:27.369 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:16:27.370 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:16:17:849478+0000", 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:16:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:27.372 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:16:17.850308+0000", 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.689+0000 7fbb40800640 1 Processor -- start 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.690+0000 7fbb40800640 1 -- start start 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.690+0000 7fbb40800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb38109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.690+0000 7fbb40800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3810a1e0 con 0x7fbb38109810 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.690+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb38109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.690+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb38109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37018/0 (socket says 172.21.15.50:37018) 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.690+0000 7fbb3de00640 1 -- 172.21.15.50:0/1350983221 learned_addr learned my addr 172.21.15.50:0/1350983221 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:27.373 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.691+0000 7fbb3de00640 1 -- 172.21.15.50:0/1350983221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3810a950 con 0x7fbb38109810 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.692+0000 7fbb3de00640 1 --2- 172.21.15.50:0/1350983221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb38109c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009820 tx=0x7fbb2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5550247aac0ec21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.692+0000 7fbb3ca00640 1 -- 172.21.15.50:0/1350983221 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c037070 con 0x7fbb38109810 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.692+0000 7fbb3ca00640 1 -- 172.21.15.50:0/1350983221 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb2c032900 con 0x7fbb38109810 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.692+0000 7fbb3ca00640 1 -- 172.21.15.50:0/1350983221 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c030850 con 0x7fbb38109810 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.693+0000 7fbb40800640 1 -- 172.21.15.50:0/1350983221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 msgr2=0x7fbb38109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.693+0000 7fbb40800640 1 --2- 172.21.15.50:0/1350983221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb38109c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009820 tx=0x7fbb2c02a270 comp rx=0 tx=0).stop 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.693+0000 7fbb40800640 1 -- 172.21.15.50:0/1350983221 shutdown_connections 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.693+0000 7fbb40800640 1 --2- 172.21.15.50:0/1350983221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb38109c10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.693+0000 7fbb40800640 1 -- 172.21.15.50:0/1350983221 >> 172.21.15.50:0/1350983221 conn(0x7fbb3807ccf0 msgr2=0x7fbb3807d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.694+0000 7fbb40800640 1 -- 172.21.15.50:0/1350983221 shutdown_connections 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.694+0000 7fbb40800640 1 -- 172.21.15.50:0/1350983221 wait complete. 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb40800640 1 Processor -- start 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb40800640 1 -- start start 2024-09-12T03:16:27.374 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb40800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb3819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb40800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3810dd00 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb3819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb3819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37026/0 (socket says 172.21.15.50:37026) 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.695+0000 7fbb3de00640 1 -- 172.21.15.50:0/918555117 learned_addr learned my addr 172.21.15.50:0/918555117 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb3de00640 1 -- 172.21.15.50:0/918555117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3819dd10 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb3de00640 1 --2- 172.21.15.50:0/918555117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb3819d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c030460 tx=0x7fbb2c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c042070 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb2c032be0 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3819dfa0 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c037040 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.696+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb381a0c90 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.697+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb2c045070 con 0x7fbb38109810 2024-09-12T03:16:27.375 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.697+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb2c047a90 con 0x7fbb38109810 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.698+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb38109c10 con 0x7fbb38109810 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.700+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbb2c032d90 con 0x7fbb38109810 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.766+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbb381a0e60 con 0x7fbb38109810 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.767+0000 7fbb32a00640 1 -- 172.21.15.50:0/918555117 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fbb2c03da10 con 0x7fbb38109810 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 msgr2=0x7fbb3819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 --2- 172.21.15.50:0/918555117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb3819d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c030460 tx=0x7fbb2c030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 shutdown_connections 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 --2- 172.21.15.50:0/918555117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb38109810 0x7fbb3819d7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 >> 172.21.15.50:0/918555117 conn(0x7fbb3807ccf0 msgr2=0x7fbb381074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 shutdown_connections 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:26.768+0000 7fbb40800640 1 -- 172.21.15.50:0/918555117 wait complete. 2024-09-12T03:16:27.376 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (1/15)... 2024-09-12T03:16:27.619 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:27 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:27.398+0000 7f4db5632040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:16:27.620 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:27 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:27.618+0000 7f4db5632040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:16:29.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:28 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:28.771+0000 7f4db5632040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:16:29.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:28 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:28.854+0000 7f4db5632040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:16:29.997 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:29 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/635249138' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:16:29.997 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:29 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:29.849+0000 7f4db5632040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:16:29.997 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:29 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:29.996+0000 7f4db5632040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:16:30.279 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.168+0000 7f4db5632040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:30.279 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.244+0000 7f4db5632040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:16:30.557 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.424+0000 7f4db5632040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:16:30.557 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.510+0000 7f4db5632040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:16:30.564 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:30.564 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:16:30.564 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:16:30.564 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:16:30.564 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:16:30.564 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:30.565 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:16:30.566 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:16:30.576 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:16:30.576 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:16:30.576 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:16:30.576 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.576 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:16:30.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:16:17:849478+0000", 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:16:30.578 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:16:30.579 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:16:17.850308+0000", 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:16:30.580 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.844+0000 7f825d800640 1 Processor -- start 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.844+0000 7f825d800640 1 -- start start 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.844+0000 7f825d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f825807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.844+0000 7f825d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f825807c8b0 con 0x7f825807de80 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.845+0000 7f8256a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f825807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.845+0000 7f8256a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f825807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35162/0 (socket says 172.21.15.50:35162) 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.845+0000 7f8256a00640 1 -- 172.21.15.50:0/3930877485 learned_addr learned my addr 172.21.15.50:0/3930877485 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.845+0000 7f8256a00640 1 -- 172.21.15.50:0/3930877485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825807ca30 con 0x7f825807de80 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.846+0000 7f8256a00640 1 --2- 172.21.15.50:0/3930877485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f825807c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8248009820 tx=0x7f824802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05a420f29e88e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.581 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.846+0000 7f8255600640 1 -- 172.21.15.50:0/3930877485 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8248037070 con 0x7f825807de80 2024-09-12T03:16:30.582 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.846+0000 7f8255600640 1 -- 172.21.15.50:0/3930877485 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8248032900 con 0x7f825807de80 2024-09-12T03:16:30.582 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.846+0000 7f8255600640 1 -- 172.21.15.50:0/3930877485 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8248030850 con 0x7f825807de80 2024-09-12T03:16:30.582 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 -- 172.21.15.50:0/3930877485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 msgr2=0x7f825807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.582 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 --2- 172.21.15.50:0/3930877485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f825807c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8248009820 tx=0x7f824802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 -- 172.21.15.50:0/3930877485 shutdown_connections 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 --2- 172.21.15.50:0/3930877485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f825807c2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 -- 172.21.15.50:0/3930877485 >> 172.21.15.50:0/3930877485 conn(0x7f8258102cc0 msgr2=0x7f8258105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 -- 172.21.15.50:0/3930877485 shutdown_connections 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.847+0000 7f825d800640 1 -- 172.21.15.50:0/3930877485 wait complete. 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.848+0000 7f825d800640 1 Processor -- start 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.848+0000 7f825d800640 1 -- start start 2024-09-12T03:16:30.583 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.849+0000 7f825d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f82581a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.849+0000 7f825d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8258109750 con 0x7f825807de80 2024-09-12T03:16:30.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.849+0000 7f8256a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f82581a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.849+0000 7f8256a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f82581a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35170/0 (socket says 172.21.15.50:35170) 2024-09-12T03:16:30.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.849+0000 7f8256a00640 1 -- 172.21.15.50:0/635249138 learned_addr learned my addr 172.21.15.50:0/635249138 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:30.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.849+0000 7f8256a00640 1 -- 172.21.15.50:0/635249138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82581a20c0 con 0x7f825807de80 2024-09-12T03:16:30.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f8256a00640 1 --2- 172.21.15.50:0/635249138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f82581a1b80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f82480097f0 tx=0x7f8248030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8248037040 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8248040070 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f824803a390 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82581a2350 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82581a5040 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f824803a630 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.850+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f82480459a0 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.851+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825807c360 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.853+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8248045460 con 0x7f825807de80 2024-09-12T03:16:30.585 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.920+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f825819b6a0 con 0x7f825807de80 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.920+0000 7f824f400640 1 -- 172.21.15.50:0/635249138 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f825819b6a0 con 0x7f825807de80 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 msgr2=0x7f82581a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 --2- 172.21.15.50:0/635249138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f82581a1b80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f82480097f0 tx=0x7f8248030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 shutdown_connections 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 --2- 172.21.15.50:0/635249138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f825807de80 0x7f82581a1b80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 >> 172.21.15.50:0/635249138 conn(0x7f8258102cc0 msgr2=0x7f8258198820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 shutdown_connections 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:29.922+0000 7f825d800640 1 -- 172.21.15.50:0/635249138 wait complete. 2024-09-12T03:16:30.586 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (2/15)... 2024-09-12T03:16:30.807 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.588+0000 7f4db5632040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:16:30.808 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.801+0000 7f4db5632040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:16:31.069 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.884+0000 7f4db5632040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:16:31.069 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:30 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:30.967+0000 7f4db5632040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:16:31.069 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:31 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:31.068+0000 7f4db5632040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:31.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:31 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:31.142+0000 7f4db5632040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:16:31.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:31 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:31.215+0000 7f4db5632040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:16:32.664 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:32.173+0000 7f4db5632040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:16:32.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: Activating manager daemon x 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: mgrmap e2: x(active, starting, since 0.00389865s) 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: Manager daemon x is now available 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' 2024-09-12T03:16:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:32 smithi050 ceph-mon[30378]: from='mgr.14100 172.21.15.50:0/3992674489' entity='mgr.x' 2024-09-12T03:16:33.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:33 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1251186725' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:16:33.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:33 smithi050 ceph-mon[30378]: mgrmap e3: x(active, since 1.0076s) 2024-09-12T03:16:33.704 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:16:17:849478+0000", 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:16:17.850308+0000", 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.016+0000 7fc53ce00640 1 Processor -- start 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.017+0000 7fc53ce00640 1 -- start start 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.017+0000 7fc53ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc538109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.017+0000 7fc53ce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810a1e0 con 0x7fc538109810 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.017+0000 7fc537400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc538109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.017+0000 7fc537400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc538109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35272/0 (socket says 172.21.15.50:35272) 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.018+0000 7fc537400640 1 -- 172.21.15.50:0/1736668609 learned_addr learned my addr 172.21.15.50:0/1736668609 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.018+0000 7fc537400640 1 -- 172.21.15.50:0/1736668609 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53810a730 con 0x7fc538109810 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.019+0000 7fc537400640 1 --2- 172.21.15.50:0/1736668609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc538109c10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc528009820 tx=0x7fc52802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8495bc4572a25022 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.019+0000 7fc536000640 1 -- 172.21.15.50:0/1736668609 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc528037070 con 0x7fc538109810 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.019+0000 7fc536000640 1 -- 172.21.15.50:0/1736668609 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc528032900 con 0x7fc538109810 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.019+0000 7fc536000640 1 -- 172.21.15.50:0/1736668609 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc528030850 con 0x7fc538109810 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 -- 172.21.15.50:0/1736668609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 msgr2=0x7fc538109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 --2- 172.21.15.50:0/1736668609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc538109c10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc528009820 tx=0x7fc52802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 -- 172.21.15.50:0/1736668609 shutdown_connections 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 --2- 172.21.15.50:0/1736668609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc538109c10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 -- 172.21.15.50:0/1736668609 >> 172.21.15.50:0/1736668609 conn(0x7fc53807cb00 msgr2=0x7fc53807cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 -- 172.21.15.50:0/1736668609 shutdown_connections 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.020+0000 7fc53ce00640 1 -- 172.21.15.50:0/1736668609 wait complete. 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.021+0000 7fc53ce00640 1 Processor -- start 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.021+0000 7fc53ce00640 1 -- start start 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.021+0000 7fc53ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc53819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.021+0000 7fc53ce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810d9f0 con 0x7fc538109810 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.022+0000 7fc537400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc53819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.022+0000 7fc537400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc53819d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35288/0 (socket says 172.21.15.50:35288) 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.022+0000 7fc537400640 1 -- 172.21.15.50:0/1251186725 learned_addr learned my addr 172.21.15.50:0/1251186725 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.022+0000 7fc537400640 1 -- 172.21.15.50:0/1251186725 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53819db10 con 0x7fc538109810 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.022+0000 7fc537400640 1 --2- 172.21.15.50:0/1251186725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc53819d5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc528031000 tx=0x7fc5280326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.023+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc528042070 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.023+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc528032be0 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.023+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc528037040 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.023+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53819dda0 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.023+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381a0a90 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.024+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 2) ==== 46043+0+0 (secure 0 0 0) 0x7fc528045070 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.024+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc538109c90 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.024+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc52806bb20 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.027+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc52802f2a0 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.092+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc5381a0c60 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.092+0000 7fc52fe00640 1 -- 172.21.15.50:0/1251186725 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc528032d90 con 0x7fc538109810 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 msgr2=0x7fc53819d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 --2- 172.21.15.50:0/1251186725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc53819d5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc528031000 tx=0x7fc5280326e0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 shutdown_connections 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 --2- 172.21.15.50:0/1251186725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc538109810 0x7fc53819d5d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 >> 172.21.15.50:0/1251186725 conn(0x7fc53807cb00 msgr2=0x7fc538107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 shutdown_connections 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:33.093+0000 7fc53ce00640 1 -- 172.21.15.50:0/1251186725 wait complete. 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (3/15)... 2024-09-12T03:16:35.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:35 smithi050 ceph-mon[30378]: mgrmap e4: x(active, since 2s) 2024-09-12T03:16:36.879 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:36 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3381056070' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:16:37.153 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:37.153 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:16:37.153 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-09-12T03:16:37.154 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:16:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:16:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:16:17:849478+0000", 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:16:17.850308+0000", 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:16:37.165 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.199+0000 7fed84600640 1 Processor -- start 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.199+0000 7fed84600640 1 -- start start 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.199+0000 7fed84600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.199+0000 7fed84600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c07c870 con 0x7fed7c07de40 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.200+0000 7fed81c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.200+0000 7fed81c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35304/0 (socket says 172.21.15.50:35304) 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.200+0000 7fed81c00640 1 -- 172.21.15.50:0/1738586914 learned_addr learned my addr 172.21.15.50:0/1738586914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.200+0000 7fed81c00640 1 -- 172.21.15.50:0/1738586914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c07c9f0 con 0x7fed7c07de40 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.201+0000 7fed81c00640 1 --2- 172.21.15.50:0/1738586914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c07c2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fed70009820 tx=0x7fed7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2a855ce68b8f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:37.166 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.201+0000 7fed7be00640 1 -- 172.21.15.50:0/1738586914 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed70037070 con 0x7fed7c07de40 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.201+0000 7fed7be00640 1 -- 172.21.15.50:0/1738586914 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed70032900 con 0x7fed7c07de40 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.201+0000 7fed7be00640 1 -- 172.21.15.50:0/1738586914 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed70030850 con 0x7fed7c07de40 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 -- 172.21.15.50:0/1738586914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 msgr2=0x7fed7c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 --2- 172.21.15.50:0/1738586914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c07c2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fed70009820 tx=0x7fed7002a270 comp rx=0 tx=0).stop 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 -- 172.21.15.50:0/1738586914 shutdown_connections 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 --2- 172.21.15.50:0/1738586914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c07c2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 -- 172.21.15.50:0/1738586914 >> 172.21.15.50:0/1738586914 conn(0x7fed7c102cc0 msgr2=0x7fed7c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 -- 172.21.15.50:0/1738586914 shutdown_connections 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.202+0000 7fed84600640 1 -- 172.21.15.50:0/1738586914 wait complete. 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.203+0000 7fed84600640 1 Processor -- start 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.203+0000 7fed84600640 1 -- start start 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed84600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed84600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c10dba0 con 0x7fed7c07de40 2024-09-12T03:16:37.167 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed81c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed81c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35306/0 (socket says 172.21.15.50:35306) 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed81c00640 1 -- 172.21.15.50:0/3381056070 learned_addr learned my addr 172.21.15.50:0/3381056070 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed81c00640 1 -- 172.21.15.50:0/3381056070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c19dc80 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.204+0000 7fed81c00640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c19d740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fed700097f0 tx=0x7fed70030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.205+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed70037040 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.205+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed70040070 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.205+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c19df10 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.205+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed7003a390 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.205+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c1a0c00 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.206+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fed7003a630 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.206+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7c07c320 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.206+0000 7fed7a000640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fed64038f20 0x7fed6403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.206+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed7006c280 con 0x7fed7c07de40 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.207+0000 7fed81200640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fed64038f20 0x7fed6403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:37.168 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.207+0000 7fed81200640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fed64038f20 0x7fed6403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed680098c0 tx=0x7fed68006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.212+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed700392f0 con 0x7fed7c07de40 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.470+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fed7c19e390 con 0x7fed7c07de40 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.470+0000 7fed7a000640 1 -- 172.21.15.50:0/3381056070 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fed7006bfe0 con 0x7fed7c07de40 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fed64038f20 msgr2=0x7fed6403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fed64038f20 0x7fed6403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed680098c0 tx=0x7fed68006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 msgr2=0x7fed7c19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c19d740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fed700097f0 tx=0x7fed70030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 shutdown_connections 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fed64038f20 0x7fed6403b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 --2- 172.21.15.50:0/3381056070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed7c07de40 0x7fed7c19d740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 >> 172.21.15.50:0/3381056070 conn(0x7fed7c102cc0 msgr2=0x7fed7c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 shutdown_connections 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:36.473+0000 7fed84600640 1 -- 172.21.15.50:0/3381056070 wait complete. 2024-09-12T03:16:37.169 INFO:teuthology.orchestra.run.smithi050.stdout:mgr is available 2024-09-12T03:16:38.144 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:37 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2330768093' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout fsid = 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:16:38.431 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.599+0000 7fc7ebe00640 1 Processor -- start 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.600+0000 7fc7ebe00640 1 -- start start 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.600+0000 7fc7ebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.600+0000 7fc7ebe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10a1e0 con 0x7fc7ec109810 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.600+0000 7fc7eaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.600+0000 7fc7eaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35320/0 (socket says 172.21.15.50:35320) 2024-09-12T03:16:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.600+0000 7fc7eaa00640 1 -- 172.21.15.50:0/2454807707 learned_addr learned my addr 172.21.15.50:0/2454807707 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.601+0000 7fc7eaa00640 1 -- 172.21.15.50:0/2454807707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec10a730 con 0x7fc7ec109810 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.602+0000 7fc7eaa00640 1 --2- 172.21.15.50:0/2454807707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec109c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009820 tx=0x7fc7dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d493272e3f99897b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.602+0000 7fc7ea000640 1 -- 172.21.15.50:0/2454807707 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc037070 con 0x7fc7ec109810 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.602+0000 7fc7ea000640 1 -- 172.21.15.50:0/2454807707 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc7dc032900 con 0x7fc7ec109810 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.602+0000 7fc7ea000640 1 -- 172.21.15.50:0/2454807707 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc030850 con 0x7fc7ec109810 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2454807707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 msgr2=0x7fc7ec109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 --2- 172.21.15.50:0/2454807707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec109c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009820 tx=0x7fc7dc02a270 comp rx=0 tx=0).stop 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2454807707 shutdown_connections 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 --2- 172.21.15.50:0/2454807707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec109c10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2454807707 >> 172.21.15.50:0/2454807707 conn(0x7fc7ec07cb00 msgr2=0x7fc7ec07cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2454807707 shutdown_connections 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.603+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2454807707 wait complete. 2024-09-12T03:16:38.433 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7ebe00640 1 Processor -- start 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7ebe00640 1 -- start start 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7ebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7ebe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10d9f0 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7eaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7eaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35330/0 (socket says 172.21.15.50:35330) 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.604+0000 7fc7eaa00640 1 -- 172.21.15.50:0/2330768093 learned_addr learned my addr 172.21.15.50:0/2330768093 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7eaa00640 1 -- 172.21.15.50:0/2330768093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec19db10 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7eaa00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec19d5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc031000 tx=0x7fc7dc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc042070 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc7dc032be0 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc037040 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec19dda0 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.605+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec1a0a90 con 0x7fc7ec109810 2024-09-12T03:16:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.606+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc7dc045070 con 0x7fc7ec109810 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.606+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec109c90 con 0x7fc7ec109810 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.607+0000 7fc7e3400640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fc7d0038f20 0x7fc7d003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.607+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc7dc06bf40 con 0x7fc7ec109810 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.607+0000 7fc7e3e00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fc7d0038f20 0x7fc7d003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.608+0000 7fc7e3e00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fc7d0038f20 0x7fc7d003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40098c0 tx=0x7fc7d4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.612+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7dc02f2a0 con 0x7fc7ec109810 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.801+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc7ec1a0c60 con 0x7fc7ec109810 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.802+0000 7fc7e3400640 1 -- 172.21.15.50:0/2330768093 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fc7dc032d90 con 0x7fc7ec109810 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fc7d0038f20 msgr2=0x7fc7d003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fc7d0038f20 0x7fc7d003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40098c0 tx=0x7fc7d4006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 msgr2=0x7fc7ec19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec19d5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc031000 tx=0x7fc7dc0326e0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.435 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 shutdown_connections 2024-09-12T03:16:38.436 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7fc7d0038f20 0x7fc7d003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.436 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 --2- 172.21.15.50:0/2330768093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7ec109810 0x7fc7ec19d5d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.436 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 >> 172.21.15.50:0/2330768093 conn(0x7fc7ec07cb00 msgr2=0x7fc7ec107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.436 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 shutdown_connections 2024-09-12T03:16:38.436 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:37.805+0000 7fc7ebe00640 1 -- 172.21.15.50:0/2330768093 wait complete. 2024-09-12T03:16:38.436 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling cephadm module... 2024-09-12T03:16:39.625 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/242392547' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T03:16:39.625 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ignoring --setuser ceph since I am not root 2024-09-12T03:16:39.625 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ignoring --setgroup ceph since I am not root 2024-09-12T03:16:39.625 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:39.625+0000 7f2521013040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:16:39.888 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:39.697+0000 7f2521013040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:40.152 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:39.888+0000 7f2521013040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:16:40.152 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:39.972+0000 7f2521013040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:16:40.152 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:40 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:40.152+0000 7f2521013040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.097+0000 7ff79a400640 1 Processor -- start 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.098+0000 7ff79a400640 1 -- start start 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.098+0000 7ff79a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff794109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.098+0000 7ff79a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff79410a1e0 con 0x7ff794109810 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.098+0000 7ff799000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff794109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.098+0000 7ff799000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff794109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35638/0 (socket says 172.21.15.50:35638) 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.098+0000 7ff799000640 1 -- 172.21.15.50:0/1237087562 learned_addr learned my addr 172.21.15.50:0/1237087562 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.099+0000 7ff799000640 1 -- 172.21.15.50:0/1237087562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79410a730 con 0x7ff794109810 2024-09-12T03:16:40.278 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff799000640 1 --2- 172.21.15.50:0/1237087562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff794109c10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff788009820 tx=0x7ff78802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdf06e23900b3e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff793400640 1 -- 172.21.15.50:0/1237087562 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff794109810 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff793400640 1 -- 172.21.15.50:0/1237087562 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff788032900 con 0x7ff794109810 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff793400640 1 -- 172.21.15.50:0/1237087562 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff788030850 con 0x7ff794109810 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff79a400640 1 -- 172.21.15.50:0/1237087562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 msgr2=0x7ff794109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff79a400640 1 --2- 172.21.15.50:0/1237087562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff794109c10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff788009820 tx=0x7ff78802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff79a400640 1 -- 172.21.15.50:0/1237087562 shutdown_connections 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.100+0000 7ff79a400640 1 --2- 172.21.15.50:0/1237087562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff794109c10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.101+0000 7ff79a400640 1 -- 172.21.15.50:0/1237087562 >> 172.21.15.50:0/1237087562 conn(0x7ff79407cb00 msgr2=0x7ff79407cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.101+0000 7ff79a400640 1 -- 172.21.15.50:0/1237087562 shutdown_connections 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.101+0000 7ff79a400640 1 -- 172.21.15.50:0/1237087562 wait complete. 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.101+0000 7ff79a400640 1 Processor -- start 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.102+0000 7ff79a400640 1 -- start start 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.102+0000 7ff79a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff79419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.279 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.102+0000 7ff79a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff79410d9f0 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.102+0000 7ff799000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff79419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.102+0000 7ff799000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff79419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35648/0 (socket says 172.21.15.50:35648) 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.102+0000 7ff799000640 1 -- 172.21.15.50:0/242392547 learned_addr learned my addr 172.21.15.50:0/242392547 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff799000640 1 -- 172.21.15.50:0/242392547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79419db10 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff799000640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff79419d5d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff788031000 tx=0x7ff7880326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff788042070 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff788032be0 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff788037040 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79419dda0 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.103+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a0a90 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.104+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff788045070 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.104+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff794109c90 con 0x7ff794109810 2024-09-12T03:16:40.280 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.105+0000 7ff791600640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7ff77c038f70 0x7ff77c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.105+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff78806bf20 con 0x7ff794109810 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.105+0000 7ff793e00640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7ff77c038f70 0x7ff77c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.106+0000 7ff793e00640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7ff77c038f70 0x7ff77c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7800098c0 tx=0x7ff780006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.109+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff788032d90 con 0x7ff794109810 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.341+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff7941a0c60 con 0x7ff794109810 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.393+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff78806b940 con 0x7ff794109810 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.394+0000 7ff791600640 1 -- 172.21.15.50:0/242392547 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff788039d80 con 0x7ff794109810 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7ff77c038f70 msgr2=0x7ff77c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7ff77c038f70 0x7ff77c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7800098c0 tx=0x7ff780006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 msgr2=0x7ff79419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff79419d5d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff788031000 tx=0x7ff7880326e0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 shutdown_connections 2024-09-12T03:16:40.282 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7ff77c038f70 0x7ff77c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.282 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 --2- 172.21.15.50:0/242392547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff794109810 0x7ff79419d5d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.282 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 >> 172.21.15.50:0/242392547 conn(0x7ff79407cb00 msgr2=0x7ff794107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.282 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 shutdown_connections 2024-09-12T03:16:40.282 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:39.396+0000 7ff79a400640 1 -- 172.21.15.50:0/242392547 wait complete. 2024-09-12T03:16:40.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:40 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:40.330+0000 7f2521013040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:16:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:40 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/242392547' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T03:16:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:40 smithi050 ceph-mon[30378]: mgrmap e5: x(active, since 7s) 2024-09-12T03:16:40.988 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:40 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:40.720+0000 7f2521013040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:16:40.988 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:40 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:40.800+0000 7f2521013040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:16:41.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:41 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:41.025+0000 7f2521013040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:16:41.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:41 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2692626961' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:16:41.981 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.095+0000 7f693c800640 1 Processor -- start 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.095+0000 7f693c800640 1 -- start start 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.095+0000 7f693c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f6934109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.095+0000 7f693c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f693410a1e0 con 0x7f6934109810 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.096+0000 7f6939e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f6934109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.096+0000 7f6939e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f6934109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35678/0 (socket says 172.21.15.50:35678) 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.096+0000 7f6939e00640 1 -- 172.21.15.50:0/2205232084 learned_addr learned my addr 172.21.15.50:0/2205232084 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.096+0000 7f6939e00640 1 -- 172.21.15.50:0/2205232084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693410a950 con 0x7f6934109810 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.097+0000 7f6939e00640 1 --2- 172.21.15.50:0/2205232084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f6934109c10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6928009820 tx=0x7f692802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1878361896b1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.098+0000 7f6938a00640 1 -- 172.21.15.50:0/2205232084 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6928037070 con 0x7f6934109810 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.098+0000 7f6938a00640 1 -- 172.21.15.50:0/2205232084 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6928032900 con 0x7f6934109810 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.098+0000 7f6938a00640 1 -- 172.21.15.50:0/2205232084 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6928030850 con 0x7f6934109810 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.098+0000 7f693c800640 1 -- 172.21.15.50:0/2205232084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 msgr2=0x7f6934109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.098+0000 7f693c800640 1 --2- 172.21.15.50:0/2205232084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f6934109c10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6928009820 tx=0x7f692802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:41.983 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.099+0000 7f693c800640 1 -- 172.21.15.50:0/2205232084 shutdown_connections 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.099+0000 7f693c800640 1 --2- 172.21.15.50:0/2205232084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f6934109c10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.099+0000 7f693c800640 1 -- 172.21.15.50:0/2205232084 >> 172.21.15.50:0/2205232084 conn(0x7f693407ccf0 msgr2=0x7f693407d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.099+0000 7f693c800640 1 -- 172.21.15.50:0/2205232084 shutdown_connections 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.099+0000 7f693c800640 1 -- 172.21.15.50:0/2205232084 wait complete. 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f693c800640 1 Processor -- start 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f693c800640 1 -- start start 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f693c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f693419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f693c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f693410dd00 con 0x7f6934109810 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f6939e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f693419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:41.984 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f6939e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f693419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35692/0 (socket says 172.21.15.50:35692) 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.100+0000 7f6939e00640 1 -- 172.21.15.50:0/2692626961 learned_addr learned my addr 172.21.15.50:0/2692626961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.101+0000 7f6939e00640 1 -- 172.21.15.50:0/2692626961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693419dd10 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.101+0000 7f6939e00640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f693419d7d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6928009940 tx=0x7f69280326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.101+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6928042070 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.101+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6928032be0 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.101+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6928037040 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.102+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693419dfa0 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.102+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69341a0c90 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.103+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6934109c10 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.103+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6928045070 con 0x7f6934109810 2024-09-12T03:16:41.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.104+0000 7f692ea00640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 0x7f691c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.104+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f692806bf70 con 0x7f6934109810 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.104+0000 7f6939400640 1 -- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 msgr2=0x7f691c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.104+0000 7f6939400640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 0x7f691c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.109+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6928032d90 con 0x7f6934109810 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.305+0000 7f6939400640 1 -- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 msgr2=0x7f691c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.305+0000 7f6939400640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 0x7f691c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.341+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f69341a1280 con 0x7f6934109810 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.342+0000 7f692ea00640 1 -- 172.21.15.50:0/2692626961 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f692806b990 con 0x7f6934109810 2024-09-12T03:16:41.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 msgr2=0x7f691c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 0x7f691c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 msgr2=0x7f693419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f693419d7d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6928009940 tx=0x7f69280326e0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 shutdown_connections 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7f691c038fc0 0x7f691c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 --2- 172.21.15.50:0/2692626961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6934109810 0x7f693419d7d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 >> 172.21.15.50:0/2692626961 conn(0x7f693407ccf0 msgr2=0x7f69341074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 shutdown_connections 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:41.344+0000 7f693c800640 1 -- 172.21.15.50:0/2692626961 wait complete. 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for the mgr to restart... 2024-09-12T03:16:41.987 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr epoch 5... 2024-09-12T03:16:42.247 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:42 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:42.192+0000 7f2521013040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:16:42.497 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:42 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:42.279+0000 7f2521013040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:16:43.381 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.239+0000 7f2521013040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:16:43.381 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.381+0000 7f2521013040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:16:43.664 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.552+0000 7f2521013040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:43.664 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.627+0000 7f2521013040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:16:44.155 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.807+0000 7f2521013040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:16:44.155 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.892+0000 7f2521013040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:16:44.156 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:43.967+0000 7f2521013040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:16:44.156 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:44.155+0000 7f2521013040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:16:44.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:44.236+0000 7f2521013040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:16:44.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:44.318+0000 7f2521013040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:16:44.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:44.420+0000 7f2521013040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:44.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:44.493+0000 7f2521013040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:16:44.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:44.567+0000 7f2521013040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:16:45.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:45.521+0000 7f2521013040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:16:45.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: Active manager daemon x restarted 2024-09-12T03:16:45.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: Activating manager daemon x 2024-09-12T03:16:45.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T03:16:45.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: mgrmap e6: x(active, starting, since 0.00361858s) 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: Manager daemon x is now available 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:45.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:45 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:46.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:46 smithi050 ceph-mon[30378]: Found migration_current of "None". Setting to last migration. 2024-09-12T03:16:46.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:46 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:16:46.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:46 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:16:46.815 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:46 smithi050 ceph-mon[30378]: mgrmap e7: x(active, since 1.00678s) 2024-09-12T03:16:47.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:16:47.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T03:16:47.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:16:47.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:16:47.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.488+0000 7efe02400640 1 Processor -- start 2024-09-12T03:16:47.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.488+0000 7efe02400640 1 -- start start 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.489+0000 7efe02400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.489+0000 7efe02400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc07c8b0 con 0x7efdfc07de80 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.489+0000 7efdfb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.489+0000 7efdfb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35706/0 (socket says 172.21.15.50:35706) 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.489+0000 7efdfb400640 1 -- 172.21.15.50:0/2029558549 learned_addr learned my addr 172.21.15.50:0/2029558549 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.489+0000 7efdfb400640 1 -- 172.21.15.50:0/2029558549 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc07ca30 con 0x7efdfc07de80 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.490+0000 7efdfb400640 1 --2- 172.21.15.50:0/2029558549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc07c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdec009820 tx=0x7efdec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a8d73670f2db11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efdfaa00640 1 -- 172.21.15.50:0/2029558549 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec037070 con 0x7efdfc07de80 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efdfaa00640 1 -- 172.21.15.50:0/2029558549 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efdec032900 con 0x7efdfc07de80 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efdfaa00640 1 -- 172.21.15.50:0/2029558549 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec030850 con 0x7efdfc07de80 2024-09-12T03:16:47.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efe02400640 1 -- 172.21.15.50:0/2029558549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 msgr2=0x7efdfc07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efe02400640 1 --2- 172.21.15.50:0/2029558549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc07c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdec009820 tx=0x7efdec02a270 comp rx=0 tx=0).stop 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efe02400640 1 -- 172.21.15.50:0/2029558549 shutdown_connections 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efe02400640 1 --2- 172.21.15.50:0/2029558549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc07c2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.491+0000 7efe02400640 1 -- 172.21.15.50:0/2029558549 >> 172.21.15.50:0/2029558549 conn(0x7efdfc102d00 msgr2=0x7efdfc105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.492+0000 7efe02400640 1 -- 172.21.15.50:0/2029558549 shutdown_connections 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.492+0000 7efe02400640 1 -- 172.21.15.50:0/2029558549 wait complete. 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.493+0000 7efe02400640 1 Processor -- start 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.493+0000 7efe02400640 1 -- start start 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efe02400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc1a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efe02400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc109770 con 0x7efdfc07de80 2024-09-12T03:16:47.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efdfb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc1a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efdfb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc1a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35708/0 (socket says 172.21.15.50:35708) 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efdfb400640 1 -- 172.21.15.50:0/3859451601 learned_addr learned my addr 172.21.15.50:0/3859451601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efdfb400640 1 -- 172.21.15.50:0/3859451601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc1a20d0 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.494+0000 7efdfb400640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc1a1b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097f0 tx=0x7efdec030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.495+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec037040 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.495+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efdec040070 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.495+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec03a390 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.495+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc1a2360 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.495+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a5050 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.497+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7efdec03b430 con 0x7efdfc07de80 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.497+0000 7efdf8c00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.497+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 --> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efde003bb40 con 0x7efde0038f70 2024-09-12T03:16:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.497+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efdec06c160 con 0x7efdfc07de80 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.497+0000 7efdf3e00640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 msgr2=0x7efde003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.497+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.698+0000 7efdf3e00640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 msgr2=0x7efde003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:42.698+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:43.099+0000 7efdf3e00640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 msgr2=0x7efde003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:43.099+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:43.900+0000 7efdf3e00640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 msgr2=0x7efde003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:43.900+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:45.500+0000 7efdf3e00640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 msgr2=0x7efde003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/1939431272 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:45.500+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-12T03:16:47.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:45.526+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7efdec032900 con 0x7efdfc07de80 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:45.526+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 msgr2=0x7efde003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:45.526+0000 7efdf8c00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.527+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7efdec05aae0 con 0x7efdfc07de80 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.528+0000 7efdf8c00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7efde003c8a0 0x7efde003ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.528+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efdec0655d0 con 0x7efde003c8a0 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.529+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7efde003c8a0 0x7efde003ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.530+0000 7efdf3e00640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7efde003c8a0 0x7efde003ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efde4003980 tx=0x7efde4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.531+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7efdec0655d0 con 0x7efde003c8a0 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.538+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7efdfc07c360 con 0x7efde003c8a0 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.539+0000 7efdf8c00640 1 -- 172.21.15.50:0/3859451601 <== mgr.14120 v2:172.21.15.50:6800/2850856182 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7efdfc07c360 con 0x7efde003c8a0 2024-09-12T03:16:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.539+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7efde003c8a0 msgr2=0x7efde003ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.539+0000 7efe02400640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7efde003c8a0 0x7efde003ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efde4003980 tx=0x7efde4009290 comp rx=0 tx=0).stop 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.539+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 msgr2=0x7efdfc1a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.539+0000 7efe02400640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc1a1b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097f0 tx=0x7efdec030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 shutdown_connections 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7efde003c8a0 0x7efde003ec90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:6800/1939431272,v1:172.21.15.50:6801/1939431272] conn(0x7efde0038f70 0x7efde003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 --2- 172.21.15.50:0/3859451601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdfc07de80 0x7efdfc1a1b90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 >> 172.21.15.50:0/3859451601 conn(0x7efdfc102d00 msgr2=0x7efdfc198840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 shutdown_connections 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:46.540+0000 7efe02400640 1 -- 172.21.15.50:0/3859451601 wait complete. 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:mgr epoch 5 is available 2024-09-12T03:16:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:Setting orchestrator backend to cephadm... 2024-09-12T03:16:48.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-mon[30378]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:16:48.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-mon[30378]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:16:48.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:48.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:48.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:48.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.651+0000 7f7a90e00640 1 Processor -- start 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.651+0000 7f7a90e00640 1 -- start start 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.651+0000 7f7a90e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c07b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.651+0000 7f7a90e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c07b780 con 0x7f7a8c07cd50 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.651+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c07b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.652+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c07b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35804/0 (socket says 172.21.15.50:35804) 2024-09-12T03:16:48.508 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.652+0000 7f7a8b400640 1 -- 172.21.15.50:0/3122920575 learned_addr learned my addr 172.21.15.50:0/3122920575 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.652+0000 7f7a8b400640 1 -- 172.21.15.50:0/3122920575 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c07b900 con 0x7f7a8c07cd50 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.653+0000 7f7a8b400640 1 --2- 172.21.15.50:0/3122920575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c07b1b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009820 tx=0x7f7a7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d9cb9aba25e551 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.653+0000 7f7a8a000640 1 -- 172.21.15.50:0/3122920575 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a74037070 con 0x7f7a8c07cd50 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.653+0000 7f7a8a000640 1 -- 172.21.15.50:0/3122920575 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a74032900 con 0x7f7a8c07cd50 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.653+0000 7f7a8a000640 1 -- 172.21.15.50:0/3122920575 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a74030850 con 0x7f7a8c07cd50 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 -- 172.21.15.50:0/3122920575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 msgr2=0x7f7a8c07b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.509 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 --2- 172.21.15.50:0/3122920575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c07b1b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009820 tx=0x7f7a7402a270 comp rx=0 tx=0).stop 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 -- 172.21.15.50:0/3122920575 shutdown_connections 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 --2- 172.21.15.50:0/3122920575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c07b1b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 -- 172.21.15.50:0/3122920575 >> 172.21.15.50:0/3122920575 conn(0x7f7a8c102b90 msgr2=0x7f7a8c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 -- 172.21.15.50:0/3122920575 shutdown_connections 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.654+0000 7f7a90e00640 1 -- 172.21.15.50:0/3122920575 wait complete. 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.655+0000 7f7a90e00640 1 Processor -- start 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.655+0000 7f7a90e00640 1 -- start start 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a90e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.510 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a90e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10d9f0 con 0x7f7a8c07cd50 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35816/0 (socket says 172.21.15.50:35816) 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a8b400640 1 -- 172.21.15.50:0/2041644673 learned_addr learned my addr 172.21.15.50:0/2041644673 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a8b400640 1 -- 172.21.15.50:0/2041644673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c19daf0 con 0x7f7a8c07cd50 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.656+0000 7f7a8b400640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c19d5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a740097f0 tx=0x7f7a74030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.657+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a74037040 con 0x7f7a8c07cd50 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.657+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a74040070 con 0x7f7a8c07cd50 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.657+0000 7f7a90e00640 1 -- 172.21.15.50:0/2041644673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c19dd80 con 0x7f7a8c07cd50 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.657+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a7403a390 con 0x7f7a8c07cd50 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.657+0000 7f7a90e00640 1 -- 172.21.15.50:0/2041644673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a0a70 con 0x7f7a8c07cd50 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.658+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c07b230 con 0x7f7a8c07cd50 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.658+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f7a74032ab0 con 0x7f7a8c07cd50 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.659+0000 7f7a83e00640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7a70038e60 0x7f7a7003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.659+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a7406c330 con 0x7f7a8c07cd50 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.659+0000 7f7a8aa00640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7a70038e60 0x7f7a7003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.660+0000 7f7a8aa00640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7a70038e60 0x7f7a7003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098c0 tx=0x7f7a7c006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.664+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a74043360 con 0x7f7a8c07cd50 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.865+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7a8c062e40 con 0x7f7a70038e60 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.871+0000 7f7a83e00640 1 -- 172.21.15.50:0/2041644673 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a8c062e40 con 0x7f7a70038e60 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7a70038e60 msgr2=0x7f7a7003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7a70038e60 0x7f7a7003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098c0 tx=0x7f7a7c006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 msgr2=0x7f7a8c19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c19d5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a740097f0 tx=0x7f7a74030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 shutdown_connections 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7a70038e60 0x7f7a7003b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 --2- 172.21.15.50:0/2041644673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a8c07cd50 0x7f7a8c19d5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.874+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 >> 172.21.15.50:0/2041644673 conn(0x7f7a8c102b90 msgr2=0x7f7a8c1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.875+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 shutdown_connections 2024-09-12T03:16:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:47.875+0000 7f7a81600640 1 -- 172.21.15.50:0/2041644673 wait complete. 2024-09-12T03:16:49.015 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:48.823+0000 7f2480600640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:16:48] ENGINE Error in HTTPServer.serve 2024-09-12T03:16:49.015 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Traceback (most recent call last): 2024-09-12T03:16:49.015 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:16:49.015 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._connections.run(self.expiration_interval) 2024-09-12T03:16:49.015 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:16:49.015 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._run(expiration_interval) 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: s = self.context.wrap_socket( 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: return self.sslsocket_class._create( 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self.do_handshake() 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._sslobj.do_handshake() 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:16:49.016 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:16:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:49 smithi050 ceph-mon[30378]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:49 smithi050 ceph-mon[30378]: mgrmap e8: x(active, since 2s) 2024-09-12T03:16:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:49 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:49.863 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T03:16:49.910 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.016+0000 7f9118800640 1 Processor -- start 2024-09-12T03:16:49.910 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.016+0000 7f9118800640 1 -- start start 2024-09-12T03:16:49.910 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.017+0000 7f9118800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f9110107950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:49.910 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.017+0000 7f9118800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9110107f20 con 0x7f9110107550 2024-09-12T03:16:49.910 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.017+0000 7f9115e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f9110107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.017+0000 7f9115e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f9110107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51414/0 (socket says 172.21.15.50:51414) 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.017+0000 7f9115e00640 1 -- 172.21.15.50:0/1753056024 learned_addr learned my addr 172.21.15.50:0/1753056024 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.017+0000 7f9115e00640 1 -- 172.21.15.50:0/1753056024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9110108750 con 0x7f9110107550 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.019+0000 7f9115e00640 1 --2- 172.21.15.50:0/1753056024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f9110107950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9104009820 tx=0x7f910402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5275049db7318ada server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.019+0000 7f9114a00640 1 -- 172.21.15.50:0/1753056024 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9104037070 con 0x7f9110107550 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.019+0000 7f9114a00640 1 -- 172.21.15.50:0/1753056024 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9104032900 con 0x7f9110107550 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.019+0000 7f9114a00640 1 -- 172.21.15.50:0/1753056024 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9104030850 con 0x7f9110107550 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 -- 172.21.15.50:0/1753056024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 msgr2=0x7f9110107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 --2- 172.21.15.50:0/1753056024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f9110107950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9104009820 tx=0x7f910402a270 comp rx=0 tx=0).stop 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 -- 172.21.15.50:0/1753056024 shutdown_connections 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 --2- 172.21.15.50:0/1753056024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f9110107950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 -- 172.21.15.50:0/1753056024 >> 172.21.15.50:0/1753056024 conn(0x7f9110102d00 msgr2=0x7f9110105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:49.911 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 -- 172.21.15.50:0/1753056024 shutdown_connections 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.020+0000 7f9118800640 1 -- 172.21.15.50:0/1753056024 wait complete. 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9118800640 1 Processor -- start 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9118800640 1 -- start start 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9118800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f911007db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9118800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f911010bb00 con 0x7f9110107550 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9115e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f911007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9115e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f911007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51416/0 (socket says 172.21.15.50:51416) 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.022+0000 7f9115e00640 1 -- 172.21.15.50:0/2854549649 learned_addr learned my addr 172.21.15.50:0/2854549649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.023+0000 7f9115e00640 1 -- 172.21.15.50:0/2854549649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911007e050 con 0x7f9110107550 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.023+0000 7f9115e00640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f911007db10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f91040097f0 tx=0x7f9104030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.023+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9104037040 con 0x7f9110107550 2024-09-12T03:16:49.912 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.023+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9104040070 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.023+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911007c240 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.023+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f910403a390 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.024+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911007c720 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.025+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9104032900 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.025+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91101079d0 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.025+0000 7f910aa00640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f90f8038fc0 0x7f90f803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.025+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f910406c4b0 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.026+0000 7f9115400640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f90f8038fc0 0x7f90f803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.026+0000 7f9115400640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f90f8038fc0 0x7f90f803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91000098c0 tx=0x7f9100006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.031+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9104042310 con 0x7f9110107550 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.218+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9110103fc0 con 0x7f90f8038fc0 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.219+0000 7f910aa00640 1 -- 172.21.15.50:0/2854549649 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9110103fc0 con 0x7f90f8038fc0 2024-09-12T03:16:49.913 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f90f8038fc0 msgr2=0x7f90f803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f90f8038fc0 0x7f90f803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91000098c0 tx=0x7f9100006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 msgr2=0x7f911007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f911007db10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f91040097f0 tx=0x7f9104030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 shutdown_connections 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f90f8038fc0 0x7f90f803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 --2- 172.21.15.50:0/2854549649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9110107550 0x7f911007db10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.221+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 >> 172.21.15.50:0/2854549649 conn(0x7f9110102d00 msgr2=0x7f91101038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.222+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 shutdown_connections 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:49.222+0000 7f9118800640 1 -- 172.21.15.50:0/2854549649 wait complete. 2024-09-12T03:16:49.914 INFO:teuthology.orchestra.run.smithi050.stdout:Generating ssh key... 2024-09-12T03:16:50.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-mon[30378]: [12/Sep/2024:03:16:48] ENGINE Bus STARTING 2024-09-12T03:16:50.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-mon[30378]: [12/Sep/2024:03:16:48] ENGINE Serving on https://172.21.15.50:7150 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-mon[30378]: [12/Sep/2024:03:16:48] ENGINE Error in HTTPServer.serve 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: Traceback (most recent call last): 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._run(expiration_interval) 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:16:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: s = self.context.wrap_socket( 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: return self.sslsocket_class._create( 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: self.do_handshake() 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._sslobj.do_handshake() 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-mon[30378]: [12/Sep/2024:03:16:48] ENGINE Serving on http://172.21.15.50:8765 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-mon[30378]: [12/Sep/2024:03:16:48] ENGINE Bus STARTED 2024-09-12T03:16:50.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-mon[30378]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:51.241 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Generating public/private rsa key pair. 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Your identification has been saved in /tmp/tmplumtkbc4/key 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Your public key has been saved in /tmp/tmplumtkbc4/key.pub 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: The key fingerprint is: 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: SHA256:Qka4DhXEQeiSN1ae2VPDZLMqFWITIHPX8Yhi2C297U8 ceph-389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: The key's randomart image is: 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: +---[RSA 3072]----+ 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: |o o*@*+++ | 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: | *.===.=+o | 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: |.o=++==.o. | 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: |o.*o=*o. | 2024-09-12T03:16:51.242 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: | + +o +.S | 2024-09-12T03:16:51.243 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: | .o . | 2024-09-12T03:16:51.243 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: | . E | 2024-09-12T03:16:51.243 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: | o | 2024-09-12T03:16:51.243 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: | . | 2024-09-12T03:16:51.243 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:16:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: +----[SHA256]-----+ 2024-09-12T03:16:51.527 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.382+0000 7fde52a00640 1 Processor -- start 2024-09-12T03:16:51.527 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.382+0000 7fde52a00640 1 -- start start 2024-09-12T03:16:51.527 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.382+0000 7fde52a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.527 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.382+0000 7fde52a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c107f00 con 0x7fde4c107530 2024-09-12T03:16:51.527 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.383+0000 7fde4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.527 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.383+0000 7fde4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51424/0 (socket says 172.21.15.50:51424) 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.383+0000 7fde4be00640 1 -- 172.21.15.50:0/3513465219 learned_addr learned my addr 172.21.15.50:0/3513465219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.383+0000 7fde4be00640 1 -- 172.21.15.50:0/3513465219 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c108730 con 0x7fde4c107530 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.384+0000 7fde4be00640 1 --2- 172.21.15.50:0/3513465219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c107930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009820 tx=0x7fde3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303174a83cfe654b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.384+0000 7fde4aa00640 1 -- 172.21.15.50:0/3513465219 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde3c037070 con 0x7fde4c107530 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.384+0000 7fde4aa00640 1 -- 172.21.15.50:0/3513465219 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde3c032900 con 0x7fde4c107530 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.384+0000 7fde4aa00640 1 -- 172.21.15.50:0/3513465219 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde3c030850 con 0x7fde4c107530 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 -- 172.21.15.50:0/3513465219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 msgr2=0x7fde4c107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 --2- 172.21.15.50:0/3513465219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c107930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009820 tx=0x7fde3c02a270 comp rx=0 tx=0).stop 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 -- 172.21.15.50:0/3513465219 shutdown_connections 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 --2- 172.21.15.50:0/3513465219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c107930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 -- 172.21.15.50:0/3513465219 >> 172.21.15.50:0/3513465219 conn(0x7fde4c102cc0 msgr2=0x7fde4c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 -- 172.21.15.50:0/3513465219 shutdown_connections 2024-09-12T03:16:51.528 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.385+0000 7fde52a00640 1 -- 172.21.15.50:0/3513465219 wait complete. 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.386+0000 7fde52a00640 1 Processor -- start 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.386+0000 7fde52a00640 1 -- start start 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.386+0000 7fde52a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.386+0000 7fde52a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c10bae0 con 0x7fde4c107530 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.387+0000 7fde4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.387+0000 7fde4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51440/0 (socket says 172.21.15.50:51440) 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.387+0000 7fde4be00640 1 -- 172.21.15.50:0/3867868652 learned_addr learned my addr 172.21.15.50:0/3867868652 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.387+0000 7fde4be00640 1 -- 172.21.15.50:0/3867868652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c19bac0 con 0x7fde4c107530 2024-09-12T03:16:51.529 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.387+0000 7fde4be00640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c19b580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097f0 tx=0x7fde3c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.388+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde3c037040 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.388+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde3c040070 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.388+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde3c03a390 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.388+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c19bd50 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.388+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c19ea40 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.389+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fde3c03b430 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.389+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c1079b0 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.390+0000 7fde48c00640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fde30038fc0 0x7fde3003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.390+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fde3c06c4e0 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.390+0000 7fde4b400640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fde30038fc0 0x7fde3003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.391+0000 7fde4b400640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fde30038fc0 0x7fde3003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fde380098c0 tx=0x7fde38006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.395+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde3c06be30 con 0x7fde4c107530 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.579+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fde4c104000 con 0x7fde30038fc0 2024-09-12T03:16:51.530 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.936+0000 7fde48c00640 1 -- 172.21.15.50:0/3867868652 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde4c104000 con 0x7fde30038fc0 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fde30038fc0 msgr2=0x7fde3003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fde30038fc0 0x7fde3003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fde380098c0 tx=0x7fde38006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 msgr2=0x7fde4c19b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c19b580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097f0 tx=0x7fde3c030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 shutdown_connections 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fde30038fc0 0x7fde3003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 --2- 172.21.15.50:0/3867868652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde4c107530 0x7fde4c19b580 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 >> 172.21.15.50:0/3867868652 conn(0x7fde4c102cc0 msgr2=0x7fde4c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 shutdown_connections 2024-09-12T03:16:51.531 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:50.939+0000 7fde52a00640 1 -- 172.21.15.50:0/3867868652 wait complete. 2024-09-12T03:16:52.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:51 smithi050 ceph-mon[30378]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:52.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:51 smithi050 ceph-mon[30378]: Generating ssh key... 2024-09-12T03:16:52.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:51 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:52.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:51 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:52.766 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:16:52.766 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.984+0000 7faa1e000640 1 Processor -- start 2024-09-12T03:16:52.766 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.984+0000 7faa1e000640 1 -- start start 2024-09-12T03:16:52.766 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.984+0000 7faa1e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa18109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.984+0000 7faa1e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa1810a1e0 con 0x7faa18109810 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.985+0000 7faa17400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa18109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.985+0000 7faa17400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa18109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51452/0 (socket says 172.21.15.50:51452) 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.985+0000 7faa17400640 1 -- 172.21.15.50:0/305882397 learned_addr learned my addr 172.21.15.50:0/305882397 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.985+0000 7faa17400640 1 -- 172.21.15.50:0/305882397 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1810a950 con 0x7faa18109810 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.986+0000 7faa17400640 1 --2- 172.21.15.50:0/305882397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa18109c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa08009820 tx=0x7faa0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21863f34632686b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.986+0000 7faa16a00640 1 -- 172.21.15.50:0/305882397 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa08037070 con 0x7faa18109810 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.986+0000 7faa16a00640 1 -- 172.21.15.50:0/305882397 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa08032900 con 0x7faa18109810 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.986+0000 7faa16a00640 1 -- 172.21.15.50:0/305882397 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa08030850 con 0x7faa18109810 2024-09-12T03:16:52.767 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 -- 172.21.15.50:0/305882397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 msgr2=0x7faa18109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 --2- 172.21.15.50:0/305882397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa18109c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa08009820 tx=0x7faa0802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 -- 172.21.15.50:0/305882397 shutdown_connections 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 --2- 172.21.15.50:0/305882397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa18109c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 -- 172.21.15.50:0/305882397 >> 172.21.15.50:0/305882397 conn(0x7faa1807ccf0 msgr2=0x7faa1807d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 -- 172.21.15.50:0/305882397 shutdown_connections 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.987+0000 7faa1e000640 1 -- 172.21.15.50:0/305882397 wait complete. 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.988+0000 7faa1e000640 1 Processor -- start 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.988+0000 7faa1e000640 1 -- start start 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.988+0000 7faa1e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa1819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.988+0000 7faa1e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa1810dd00 con 0x7faa18109810 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa17400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa1819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa17400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa1819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51462/0 (socket says 172.21.15.50:51462) 2024-09-12T03:16:52.768 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa17400640 1 -- 172.21.15.50:0/678830062 learned_addr learned my addr 172.21.15.50:0/678830062 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa17400640 1 -- 172.21.15.50:0/678830062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1819dd10 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa17400640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa1819d7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa08030460 tx=0x7faa08030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa08042070 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa08032be0 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa08037040 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.989+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1819dfa0 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.990+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a0c90 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.991+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faa08045070 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.991+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa18109c10 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.991+0000 7faa14c00640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fa9fc038fc0 0x7fa9fc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.991+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa0806bed0 con 0x7faa18109810 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.991+0000 7faa0fe00640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fa9fc038fc0 0x7fa9fc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.769 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.992+0000 7faa0fe00640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fa9fc038fc0 0x7fa9fc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa000098c0 tx=0x7faa00006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:51.996+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa08045360 con 0x7faa18109810 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.181+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7faa18107be0 con 0x7fa9fc038fc0 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.182+0000 7faa14c00640 1 -- 172.21.15.50:0/678830062 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7faa18107be0 con 0x7fa9fc038fc0 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.184+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fa9fc038fc0 msgr2=0x7fa9fc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.184+0000 7faa1e000640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fa9fc038fc0 0x7fa9fc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa000098c0 tx=0x7faa00006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.184+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 msgr2=0x7faa1819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.184+0000 7faa1e000640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa1819d7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa08030460 tx=0x7faa08030d80 comp rx=0 tx=0).stop 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.185+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 shutdown_connections 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.185+0000 7faa1e000640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fa9fc038fc0 0x7fa9fc03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.185+0000 7faa1e000640 1 --2- 172.21.15.50:0/678830062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa18109810 0x7faa1819d7d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.185+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 >> 172.21.15.50:0/678830062 conn(0x7faa1807ccf0 msgr2=0x7faa181074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.185+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 shutdown_connections 2024-09-12T03:16:52.770 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:52.185+0000 7faa1e000640 1 -- 172.21.15.50:0/678830062 wait complete. 2024-09-12T03:16:52.771 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:16:52.771 INFO:teuthology.orchestra.run.smithi050.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T03:16:52.771 INFO:teuthology.orchestra.run.smithi050.stdout:Adding host smithi050... 2024-09-12T03:16:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:52 smithi050 ceph-mon[30378]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:54.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:53 smithi050 ceph-mon[30378]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "addr": "172.21.15.50", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:55.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:55 smithi050 ceph-mon[30378]: Deploying cephadm binary to smithi050 2024-09-12T03:16:58.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Added host 'smithi050' with addr '172.21.15.50' 2024-09-12T03:16:58.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.219+0000 7fb667a00640 1 Processor -- start 2024-09-12T03:16:58.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.219+0000 7fb667a00640 1 -- start start 2024-09-12T03:16:58.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.220+0000 7fb667a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb660107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.220+0000 7fb667a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb660107f00 con 0x7fb660107530 2024-09-12T03:16:58.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.220+0000 7fb665000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb660107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.220+0000 7fb665000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb660107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51464/0 (socket says 172.21.15.50:51464) 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.220+0000 7fb665000640 1 -- 172.21.15.50:0/2956318098 learned_addr learned my addr 172.21.15.50:0/2956318098 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.221+0000 7fb665000640 1 -- 172.21.15.50:0/2956318098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660108730 con 0x7fb660107530 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.221+0000 7fb665000640 1 --2- 172.21.15.50:0/2956318098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb660107930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb654009820 tx=0x7fb65402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f7638625731037 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb65f400640 1 -- 172.21.15.50:0/2956318098 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb654037070 con 0x7fb660107530 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb65f400640 1 -- 172.21.15.50:0/2956318098 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb654032900 con 0x7fb660107530 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb65f400640 1 -- 172.21.15.50:0/2956318098 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb654030850 con 0x7fb660107530 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb667a00640 1 -- 172.21.15.50:0/2956318098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 msgr2=0x7fb660107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb667a00640 1 --2- 172.21.15.50:0/2956318098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb660107930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb654009820 tx=0x7fb65402a270 comp rx=0 tx=0).stop 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb667a00640 1 -- 172.21.15.50:0/2956318098 shutdown_connections 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb667a00640 1 --2- 172.21.15.50:0/2956318098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb660107930 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.222+0000 7fb667a00640 1 -- 172.21.15.50:0/2956318098 >> 172.21.15.50:0/2956318098 conn(0x7fb660102cc0 msgr2=0x7fb660105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.618 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.223+0000 7fb667a00640 1 -- 172.21.15.50:0/2956318098 shutdown_connections 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.223+0000 7fb667a00640 1 -- 172.21.15.50:0/2956318098 wait complete. 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.223+0000 7fb667a00640 1 Processor -- start 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb667a00640 1 -- start start 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb667a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb66019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb667a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010bae0 con 0x7fb660107530 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb665000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb66019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb665000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb66019b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51472/0 (socket says 172.21.15.50:51472) 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb665000640 1 -- 172.21.15.50:0/798484479 learned_addr learned my addr 172.21.15.50:0/798484479 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.224+0000 7fb665000640 1 -- 172.21.15.50:0/798484479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66019bac0 con 0x7fb660107530 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.225+0000 7fb665000640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb66019b580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb654009940 tx=0x7fb654032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.225+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb654042070 con 0x7fb660107530 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.225+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6540308c0 con 0x7fb660107530 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.225+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb654037040 con 0x7fb660107530 2024-09-12T03:16:58.619 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.226+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66019bd50 con 0x7fb660107530 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.226+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66019ea40 con 0x7fb660107530 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.227+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb654045070 con 0x7fb660107530 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.227+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66010bc60 con 0x7fb660107530 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.227+0000 7fb65d600640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fb648038f70 0x7fb64803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.227+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb65406c0d0 con 0x7fb660107530 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.227+0000 7fb65fe00640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fb648038f70 0x7fb64803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.228+0000 7fb65fe00640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fb648038f70 0x7fb64803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0098c0 tx=0x7fb64c006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.232+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6540371e0 con 0x7fb660107530 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:53.423+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "addr": "172.21.15.50", "target": ["mon-mgr", ""]}) -- 0x7fb660104000 con 0x7fb648038f70 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.963+0000 7fb65d600640 1 -- 172.21.15.50:0/798484479 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb660104000 con 0x7fb648038f70 2024-09-12T03:16:58.620 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fb648038f70 msgr2=0x7fb64803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fb648038f70 0x7fb64803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0098c0 tx=0x7fb64c006db0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 msgr2=0x7fb66019b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb66019b580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb654009940 tx=0x7fb654032ce0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 shutdown_connections 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fb648038f70 0x7fb64803b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 --2- 172.21.15.50:0/798484479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb660107530 0x7fb66019b580 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 >> 172.21.15.50:0/798484479 conn(0x7fb660102cc0 msgr2=0x7fb6601038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 shutdown_connections 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:57.966+0000 7fb667a00640 1 -- 172.21.15.50:0/798484479 wait complete. 2024-09-12T03:16:58.621 INFO:teuthology.orchestra.run.smithi050.stdout:Deploying unmanaged mon service... 2024-09-12T03:16:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:58 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:16:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:58 smithi050 ceph-mon[30378]: Added host smithi050 2024-09-12T03:16:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:16:58 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:00.021 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T03:17:00.021 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.187+0000 7fc645000640 1 Processor -- start 2024-09-12T03:17:00.021 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.187+0000 7fc645000640 1 -- start start 2024-09-12T03:17:00.021 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.188+0000 7fc645000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.021 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.188+0000 7fc645000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc64007c870 con 0x7fc64007de40 2024-09-12T03:17:00.022 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.188+0000 7fc63de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.022 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.188+0000 7fc63de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37428/0 (socket says 172.21.15.50:37428) 2024-09-12T03:17:00.022 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.188+0000 7fc63de00640 1 -- 172.21.15.50:0/1144702961 learned_addr learned my addr 172.21.15.50:0/1144702961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:00.022 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.188+0000 7fc63de00640 1 -- 172.21.15.50:0/1144702961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64007c9f0 con 0x7fc64007de40 2024-09-12T03:17:00.026 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.189+0000 7fc63de00640 1 --2- 172.21.15.50:0/1144702961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64007c2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc630009820 tx=0x7fc63002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2550cf2fa361dbce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.026 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc63d400640 1 -- 172.21.15.50:0/1144702961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630037070 con 0x7fc64007de40 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc63d400640 1 -- 172.21.15.50:0/1144702961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc630032900 con 0x7fc64007de40 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc63d400640 1 -- 172.21.15.50:0/1144702961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630030850 con 0x7fc64007de40 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 -- 172.21.15.50:0/1144702961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 msgr2=0x7fc64007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 --2- 172.21.15.50:0/1144702961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64007c2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc630009820 tx=0x7fc63002a270 comp rx=0 tx=0).stop 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 -- 172.21.15.50:0/1144702961 shutdown_connections 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 --2- 172.21.15.50:0/1144702961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64007c2a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 -- 172.21.15.50:0/1144702961 >> 172.21.15.50:0/1144702961 conn(0x7fc640102cc0 msgr2=0x7fc640105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 -- 172.21.15.50:0/1144702961 shutdown_connections 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.190+0000 7fc645000640 1 -- 172.21.15.50:0/1144702961 wait complete. 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.191+0000 7fc645000640 1 Processor -- start 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.191+0000 7fc645000640 1 -- start start 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.192+0000 7fc645000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.027 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.192+0000 7fc645000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010dba0 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.192+0000 7fc63de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.192+0000 7fc63de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37440/0 (socket says 172.21.15.50:37440) 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.192+0000 7fc63de00640 1 -- 172.21.15.50:0/183693703 learned_addr learned my addr 172.21.15.50:0/183693703 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.192+0000 7fc63de00640 1 -- 172.21.15.50:0/183693703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64019dc80 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.193+0000 7fc63de00640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64019d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc630032ce0 tx=0x7fc630032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.193+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630042070 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.193+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6300308c0 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.193+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64019df10 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.193+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc630037040 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.193+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6401a0c00 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.194+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc630045070 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.194+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc64007c320 con 0x7fc64007de40 2024-09-12T03:17:00.028 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.195+0000 7fc637400640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fc624038fc0 0x7fc62403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.195+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc63006c090 con 0x7fc64007de40 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.195+0000 7fc635400640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fc624038fc0 0x7fc62403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.195+0000 7fc635400640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fc624038fc0 0x7fc62403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098c0 tx=0x7fc628006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.200+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc63006b9e0 con 0x7fc64007de40 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.421+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc640103f80 con 0x7fc624038fc0 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.426+0000 7fc637400640 1 -- 172.21.15.50:0/183693703 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc640103f80 con 0x7fc624038fc0 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fc624038fc0 msgr2=0x7fc62403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fc624038fc0 0x7fc62403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098c0 tx=0x7fc628006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 msgr2=0x7fc64019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64019d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc630032ce0 tx=0x7fc630032d10 comp rx=0 tx=0).stop 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 shutdown_connections 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fc624038fc0 0x7fc62403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.029 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 --2- 172.21.15.50:0/183693703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64007de40 0x7fc64019d740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.030 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 >> 172.21.15.50:0/183693703 conn(0x7fc640102cc0 msgr2=0x7fc640103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.030 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 shutdown_connections 2024-09-12T03:17:00.030 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:16:59.429+0000 7fc645000640 1 -- 172.21.15.50:0/183693703 wait complete. 2024-09-12T03:17:00.030 INFO:teuthology.orchestra.run.smithi050.stdout:Deploying unmanaged mgr service... 2024-09-12T03:17:00.536 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:00 smithi050 ceph-mon[30378]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:00.536 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:00 smithi050 ceph-mon[30378]: Saving service mon spec with placement count:5 2024-09-12T03:17:00.536 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:00 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:17:02.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:02 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:17:02.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:02 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:17:02.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:02 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:17:02.438 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T03:17:02.438 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.598+0000 7fd834e00640 1 Processor -- start 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.598+0000 7fd834e00640 1 -- start start 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.598+0000 7fd834e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd830109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.598+0000 7fd834e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010a1e0 con 0x7fd830109810 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.598+0000 7fd82de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd830109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.598+0000 7fd82de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd830109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37450/0 (socket says 172.21.15.50:37450) 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.599+0000 7fd82de00640 1 -- 172.21.15.50:0/3400882912 learned_addr learned my addr 172.21.15.50:0/3400882912 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.599+0000 7fd82de00640 1 -- 172.21.15.50:0/3400882912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83010a950 con 0x7fd830109810 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.600+0000 7fd82de00640 1 --2- 172.21.15.50:0/3400882912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd830109c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd820009820 tx=0x7fd82002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0db241022d05568 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.600+0000 7fd82d400640 1 -- 172.21.15.50:0/3400882912 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820037070 con 0x7fd830109810 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.600+0000 7fd82d400640 1 -- 172.21.15.50:0/3400882912 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd820032920 con 0x7fd830109810 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.600+0000 7fd82d400640 1 -- 172.21.15.50:0/3400882912 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820030810 con 0x7fd830109810 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 -- 172.21.15.50:0/3400882912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 msgr2=0x7fd830109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.439 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 --2- 172.21.15.50:0/3400882912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd830109c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd820009820 tx=0x7fd82002a270 comp rx=0 tx=0).stop 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 -- 172.21.15.50:0/3400882912 shutdown_connections 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 --2- 172.21.15.50:0/3400882912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd830109c10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 -- 172.21.15.50:0/3400882912 >> 172.21.15.50:0/3400882912 conn(0x7fd83007ccf0 msgr2=0x7fd83007d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 -- 172.21.15.50:0/3400882912 shutdown_connections 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.601+0000 7fd834e00640 1 -- 172.21.15.50:0/3400882912 wait complete. 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.602+0000 7fd834e00640 1 Processor -- start 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.602+0000 7fd834e00640 1 -- start start 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.603+0000 7fd834e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd8301a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.603+0000 7fd834e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010dd00 con 0x7fd830109810 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.603+0000 7fd82de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd8301a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.603+0000 7fd82de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd8301a1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37466/0 (socket says 172.21.15.50:37466) 2024-09-12T03:17:02.440 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.603+0000 7fd82de00640 1 -- 172.21.15.50:0/2437958736 learned_addr learned my addr 172.21.15.50:0/2437958736 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.603+0000 7fd82de00640 1 -- 172.21.15.50:0/2437958736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301a2100 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.604+0000 7fd82de00640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd8301a1bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd8200097f0 tx=0x7fd8200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.604+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820037040 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.604+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd820030c40 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.604+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd82003a5d0 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.604+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a2390 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.604+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a5080 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.606+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd82003b3f0 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.606+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd830109c90 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.606+0000 7fd827400640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fd814038f70 0x7fd81403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.606+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd82006c470 con 0x7fd830109810 2024-09-12T03:17:02.441 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.606+0000 7fd825400640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fd814038f70 0x7fd81403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.607+0000 7fd825400640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fd814038f70 0x7fd81403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098c0 tx=0x7fd818006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.612+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd82003b6e0 con 0x7fd830109810 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.833+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd830107c70 con 0x7fd814038f70 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.838+0000 7fd827400640 1 -- 172.21.15.50:0/2437958736 <== mgr.14120 v2:172.21.15.50:6800/2850856182 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd830107c70 con 0x7fd814038f70 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fd814038f70 msgr2=0x7fd81403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fd814038f70 0x7fd81403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098c0 tx=0x7fd818006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 msgr2=0x7fd8301a1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd8301a1bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd8200097f0 tx=0x7fd8200327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 shutdown_connections 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7fd814038f70 0x7fd81403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 --2- 172.21.15.50:0/2437958736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd830109810 0x7fd8301a1bc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.840+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 >> 172.21.15.50:0/2437958736 conn(0x7fd83007ccf0 msgr2=0x7fd830107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:02.442 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.841+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 shutdown_connections 2024-09-12T03:17:02.443 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:01.841+0000 7fd834e00640 1 -- 172.21.15.50:0/2437958736 wait complete. 2024-09-12T03:17:03.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:03 smithi050 ceph-mon[30378]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:03.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:03 smithi050 ceph-mon[30378]: Saving service mgr spec with placement count:2 2024-09-12T03:17:03.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:03 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/76971093' entity='client.admin' 2024-09-12T03:17:03.818 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.936+0000 7f4521200640 1 Processor -- start 2024-09-12T03:17:03.818 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.936+0000 7f4521200640 1 -- start start 2024-09-12T03:17:03.818 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.936+0000 7f4521200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.818 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.936+0000 7f4521200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f451c10a1e0 con 0x7f451c109810 2024-09-12T03:17:03.818 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.937+0000 7f4519e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.819 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.937+0000 7f4519e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37476/0 (socket says 172.21.15.50:37476) 2024-09-12T03:17:03.819 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.937+0000 7f4519e00640 1 -- 172.21.15.50:0/2121515612 learned_addr learned my addr 172.21.15.50:0/2121515612 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:03.819 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.937+0000 7f4519e00640 1 -- 172.21.15.50:0/2121515612 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c10a950 con 0x7f451c109810 2024-09-12T03:17:03.819 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.938+0000 7f4519e00640 1 --2- 172.21.15.50:0/2121515612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c109c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f450c009820 tx=0x7f450c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdff2e8d00adb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:03.819 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.938+0000 7f4518a00640 1 -- 172.21.15.50:0/2121515612 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f450c037070 con 0x7f451c109810 2024-09-12T03:17:03.823 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.938+0000 7f4518a00640 1 -- 172.21.15.50:0/2121515612 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c032920 con 0x7f451c109810 2024-09-12T03:17:03.824 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.938+0000 7f4518a00640 1 -- 172.21.15.50:0/2121515612 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f450c030810 con 0x7f451c109810 2024-09-12T03:17:03.824 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 -- 172.21.15.50:0/2121515612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 msgr2=0x7f451c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.824 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 --2- 172.21.15.50:0/2121515612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c109c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f450c009820 tx=0x7f450c02a270 comp rx=0 tx=0).stop 2024-09-12T03:17:03.824 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 -- 172.21.15.50:0/2121515612 shutdown_connections 2024-09-12T03:17:03.824 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 --2- 172.21.15.50:0/2121515612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c109c10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 -- 172.21.15.50:0/2121515612 >> 172.21.15.50:0/2121515612 conn(0x7f451c07ccf0 msgr2=0x7f451c07d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:03.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 -- 172.21.15.50:0/2121515612 shutdown_connections 2024-09-12T03:17:03.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.939+0000 7f4521200640 1 -- 172.21.15.50:0/2121515612 wait complete. 2024-09-12T03:17:03.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.940+0000 7f4521200640 1 Processor -- start 2024-09-12T03:17:03.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.940+0000 7f4521200640 1 -- start start 2024-09-12T03:17:03.825 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.940+0000 7f4521200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.826 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.941+0000 7f4521200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f451c10dd00 con 0x7f451c109810 2024-09-12T03:17:03.826 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.941+0000 7f4519e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.826 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.941+0000 7f4519e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37492/0 (socket says 172.21.15.50:37492) 2024-09-12T03:17:03.826 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.941+0000 7f4519e00640 1 -- 172.21.15.50:0/76971093 learned_addr learned my addr 172.21.15.50:0/76971093 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:03.826 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.941+0000 7f4519e00640 1 -- 172.21.15.50:0/76971093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c19dcf0 con 0x7f451c109810 2024-09-12T03:17:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.941+0000 7f4519e00640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c19d7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f450c030420 tx=0x7f450c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.942+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f450c037040 con 0x7f451c109810 2024-09-12T03:17:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.942+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c040070 con 0x7f451c109810 2024-09-12T03:17:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.942+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c19df80 con 0x7f451c109810 2024-09-12T03:17:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.942+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f450c03a6e0 con 0x7f451c109810 2024-09-12T03:17:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.942+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1a0c70 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.943+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f450c03a9e0 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.943+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c109c90 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.943+0000 7f4512a00640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f4500038f70 0x7f450003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.943+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f450c06c670 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.944+0000 7f4519400640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f4500038f70 0x7f450003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.944+0000 7f4519400640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f4500038f70 0x7f450003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f45080098c0 tx=0x7f4508006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:02.948+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f450c043940 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.136+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f451c1a1270 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.138+0000 7f4512a00640 1 -- 172.21.15.50:0/76971093 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f450c06bfe0 con 0x7f451c109810 2024-09-12T03:17:03.828 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f4500038f70 msgr2=0x7f450003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f4500038f70 0x7f450003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f45080098c0 tx=0x7f4508006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 msgr2=0x7f451c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c19d7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f450c030420 tx=0x7f450c032670 comp rx=0 tx=0).stop 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 shutdown_connections 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f4500038f70 0x7f450003b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 --2- 172.21.15.50:0/76971093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f451c109810 0x7f451c19d7b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 >> 172.21.15.50:0/76971093 conn(0x7f451c07ccf0 msgr2=0x7f451c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 shutdown_connections 2024-09-12T03:17:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:03.141+0000 7f4521200640 1 -- 172.21.15.50:0/76971093 wait complete. 2024-09-12T03:17:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.336+0000 7f5040e00640 1 Processor -- start 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.336+0000 7f5040e00640 1 -- start start 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.336+0000 7f5040e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c1099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.336+0000 7f5040e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f503c109f70 con 0x7f503c1095a0 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.337+0000 7f503b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c1099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.337+0000 7f503b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c1099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37496/0 (socket says 172.21.15.50:37496) 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.337+0000 7f503b400640 1 -- 172.21.15.50:0/4152863540 learned_addr learned my addr 172.21.15.50:0/4152863540 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.337+0000 7f503b400640 1 -- 172.21.15.50:0/4152863540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c10a7a0 con 0x7f503c1095a0 2024-09-12T03:17:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.338+0000 7f503b400640 1 --2- 172.21.15.50:0/4152863540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c1099a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f502c009820 tx=0x7f502c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756d87e1fe33ae06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.338+0000 7f503a000640 1 -- 172.21.15.50:0/4152863540 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f502c037070 con 0x7f503c1095a0 2024-09-12T03:17:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.338+0000 7f503a000640 1 -- 172.21.15.50:0/4152863540 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c032920 con 0x7f503c1095a0 2024-09-12T03:17:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.338+0000 7f503a000640 1 -- 172.21.15.50:0/4152863540 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f502c030810 con 0x7f503c1095a0 2024-09-12T03:17:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 -- 172.21.15.50:0/4152863540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 msgr2=0x7f503c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 --2- 172.21.15.50:0/4152863540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c1099a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f502c009820 tx=0x7f502c02a270 comp rx=0 tx=0).stop 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 -- 172.21.15.50:0/4152863540 shutdown_connections 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 --2- 172.21.15.50:0/4152863540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c1099a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 -- 172.21.15.50:0/4152863540 >> 172.21.15.50:0/4152863540 conn(0x7f503c07cbc0 msgr2=0x7f503c107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 -- 172.21.15.50:0/4152863540 shutdown_connections 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.339+0000 7f5040e00640 1 -- 172.21.15.50:0/4152863540 wait complete. 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.340+0000 7f5040e00640 1 Processor -- start 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.340+0000 7f5040e00640 1 -- start start 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.340+0000 7f5040e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.340+0000 7f5040e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f503c10db50 con 0x7f503c1095a0 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.341+0000 7f503b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.341+0000 7f503b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37504/0 (socket says 172.21.15.50:37504) 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.341+0000 7f503b400640 1 -- 172.21.15.50:0/180433954 learned_addr learned my addr 172.21.15.50:0/180433954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.341+0000 7f503b400640 1 -- 172.21.15.50:0/180433954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c19dae0 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.341+0000 7f503b400640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c19d5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f502c0097f0 tx=0x7f502c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.342+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f502c037040 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.342+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c030cc0 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.342+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f502c03a640 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.342+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c19dd70 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.342+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c1a0a60 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.343+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f502c03b3f0 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.343+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503c10dcd0 con 0x7f503c1095a0 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.343+0000 7f5033e00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5020038fc0 0x7f502003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.343+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f502c06c570 con 0x7f503c1095a0 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.344+0000 7f503aa00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5020038fc0 0x7f502003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.344+0000 7f503aa00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5020038fc0 0x7f502003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f50280098c0 tx=0x7f5028006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.349+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f502c042e10 con 0x7f503c1095a0 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.533+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f503c109a20 con 0x7f503c1095a0 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.536+0000 7f5033e00640 1 -- 172.21.15.50:0/180433954 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f502c0328c0 con 0x7f503c1095a0 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5020038fc0 msgr2=0x7f502003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5020038fc0 0x7f502003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f50280098c0 tx=0x7f5028006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 msgr2=0x7f503c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c19d5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f502c0097f0 tx=0x7f502c0327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 shutdown_connections 2024-09-12T03:17:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5020038fc0 0x7f502003b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 --2- 172.21.15.50:0/180433954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f503c1095a0 0x7f503c19d5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.538+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 >> 172.21.15.50:0/180433954 conn(0x7f503c07cbc0 msgr2=0x7f503c106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.539+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 shutdown_connections 2024-09-12T03:17:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:04.539+0000 7f5040e00640 1 -- 172.21.15.50:0/180433954 wait complete. 2024-09-12T03:17:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling the dashboard module... 2024-09-12T03:17:05.584 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:05 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/180433954' entity='client.admin' 2024-09-12T03:17:05.584 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:05 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:17:06.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-mon[30378]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:06.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2231550146' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T03:17:06.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-mon[30378]: from='mgr.14120 172.21.15.50:0/4162240829' entity='mgr.x' 2024-09-12T03:17:06.791 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ignoring --setuser ceph since I am not root 2024-09-12T03:17:06.791 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ignoring --setgroup ceph since I am not root 2024-09-12T03:17:06.791 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:06.789+0000 7f453de13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:17:07.052 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:06.861+0000 7f453de13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:17:07.052 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:07.051+0000 7f453de13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:17:07.141 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.804+0000 7f5c58000640 1 Processor -- start 2024-09-12T03:17:07.144 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.804+0000 7f5c58000640 1 -- start start 2024-09-12T03:17:07.144 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.804+0000 7f5c58000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c50109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.144 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.804+0000 7f5c58000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5010a1e0 con 0x7f5c50109810 2024-09-12T03:17:07.144 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.805+0000 7f5c55600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c50109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.144 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.805+0000 7f5c55600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c50109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37516/0 (socket says 172.21.15.50:37516) 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.805+0000 7f5c55600640 1 -- 172.21.15.50:0/3654404218 learned_addr learned my addr 172.21.15.50:0/3654404218 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.805+0000 7f5c55600640 1 -- 172.21.15.50:0/3654404218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5010a950 con 0x7f5c50109810 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.806+0000 7f5c55600640 1 --2- 172.21.15.50:0/3654404218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c50109c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009820 tx=0x7f5c4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf20ae44f8c2cff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.807+0000 7f5c4fe00640 1 -- 172.21.15.50:0/3654404218 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44037070 con 0x7f5c50109810 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.807+0000 7f5c4fe00640 1 -- 172.21.15.50:0/3654404218 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44032920 con 0x7f5c50109810 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.807+0000 7f5c4fe00640 1 -- 172.21.15.50:0/3654404218 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44030810 con 0x7f5c50109810 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.807+0000 7f5c58000640 1 -- 172.21.15.50:0/3654404218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 msgr2=0x7f5c50109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.808+0000 7f5c58000640 1 --2- 172.21.15.50:0/3654404218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c50109c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009820 tx=0x7f5c4402a270 comp rx=0 tx=0).stop 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.808+0000 7f5c58000640 1 -- 172.21.15.50:0/3654404218 shutdown_connections 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.808+0000 7f5c58000640 1 --2- 172.21.15.50:0/3654404218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c50109c10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.808+0000 7f5c58000640 1 -- 172.21.15.50:0/3654404218 >> 172.21.15.50:0/3654404218 conn(0x7f5c5007ccf0 msgr2=0x7f5c5007d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.808+0000 7f5c58000640 1 -- 172.21.15.50:0/3654404218 shutdown_connections 2024-09-12T03:17:07.145 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.808+0000 7f5c58000640 1 -- 172.21.15.50:0/3654404218 wait complete. 2024-09-12T03:17:07.149 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c58000640 1 Processor -- start 2024-09-12T03:17:07.149 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c58000640 1 -- start start 2024-09-12T03:17:07.149 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c58000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c5019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.149 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c58000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5010dd00 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c55600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c5019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c55600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c5019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37530/0 (socket says 172.21.15.50:37530) 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.809+0000 7f5c55600640 1 -- 172.21.15.50:0/2231550146 learned_addr learned my addr 172.21.15.50:0/2231550146 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.810+0000 7f5c55600640 1 -- 172.21.15.50:0/2231550146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5019dcf0 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.810+0000 7f5c55600640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c5019d7b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097f0 tx=0x7f5c440327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.810+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c44037040 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.811+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5019df80 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.811+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44030c40 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.811+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c4403a5d0 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.811+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c501a0c70 con 0x7f5c50109810 2024-09-12T03:17:07.150 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.812+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5c4403a810 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.812+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c50109c90 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.812+0000 7f5c4e000640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5c38038f70 0x7f5c3803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.812+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c4406c530 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.813+0000 7f5c54c00640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5c38038f70 0x7f5c3803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.813+0000 7f5c54c00640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5c38038f70 0x7f5c3803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0098c0 tx=0x7f5c3c006db0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:05.817+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c440371e0 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.054+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5c501a1260 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.539+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5c44030df0 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.541+0000 7f5c4e000640 1 -- 172.21.15.50:0/2231550146 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5c44044920 con 0x7f5c50109810 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.542+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5c38038f70 msgr2=0x7f5c3803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.542+0000 7f5c58000640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5c38038f70 0x7f5c3803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0098c0 tx=0x7f5c3c006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.542+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 msgr2=0x7f5c5019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.542+0000 7f5c58000640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c5019d7b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097f0 tx=0x7f5c440327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.543+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 shutdown_connections 2024-09-12T03:17:07.152 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.543+0000 7f5c58000640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f5c38038f70 0x7f5c3803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.152 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.543+0000 7f5c58000640 1 --2- 172.21.15.50:0/2231550146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c50109810 0x7f5c5019d7b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.152 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.543+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 >> 172.21.15.50:0/2231550146 conn(0x7f5c5007ccf0 msgr2=0x7f5c50107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:07.152 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.543+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 shutdown_connections 2024-09-12T03:17:07.152 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:06.543+0000 7f5c58000640 1 -- 172.21.15.50:0/2231550146 wait complete. 2024-09-12T03:17:07.315 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:07.133+0000 7f453de13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:17:07.316 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:07.314+0000 7f453de13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:17:07.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2231550146' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T03:17:07.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-mon[30378]: mgrmap e9: x(active, since 21s) 2024-09-12T03:17:07.630 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:07.491+0000 7f453de13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:17:08.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:07.884+0000 7f453de13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:17:08.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:07.967+0000 7f453de13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:17:08.542 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:08.186+0000 7f453de13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:17:08.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:08 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1489088847' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.650+0000 7f8a6c000640 1 Processor -- start 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.650+0000 7f8a6c000640 1 -- start start 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.651+0000 7f8a6c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a64109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.651+0000 7f8a6c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410a1a0 con 0x7f8a641097d0 2024-09-12T03:17:09.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.651+0000 7f8a69600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a64109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.651+0000 7f8a69600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a64109bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37546/0 (socket says 172.21.15.50:37546) 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.651+0000 7f8a69600640 1 -- 172.21.15.50:0/1882540975 learned_addr learned my addr 172.21.15.50:0/1882540975 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.652+0000 7f8a69600640 1 -- 172.21.15.50:0/1882540975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6410a920 con 0x7f8a641097d0 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.653+0000 7f8a69600640 1 --2- 172.21.15.50:0/1882540975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a64109bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009820 tx=0x7f8a5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e032e282d95682 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.653+0000 7f8a63e00640 1 -- 172.21.15.50:0/1882540975 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a58037070 con 0x7f8a641097d0 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.653+0000 7f8a63e00640 1 -- 172.21.15.50:0/1882540975 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a58032920 con 0x7f8a641097d0 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.653+0000 7f8a63e00640 1 -- 172.21.15.50:0/1882540975 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a58030810 con 0x7f8a641097d0 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 -- 172.21.15.50:0/1882540975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 msgr2=0x7f8a64109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 --2- 172.21.15.50:0/1882540975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a64109bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009820 tx=0x7f8a5802a270 comp rx=0 tx=0).stop 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 -- 172.21.15.50:0/1882540975 shutdown_connections 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 --2- 172.21.15.50:0/1882540975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a64109bd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 -- 172.21.15.50:0/1882540975 >> 172.21.15.50:0/1882540975 conn(0x7f8a6407ccf0 msgr2=0x7f8a6407d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:09.285 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 -- 172.21.15.50:0/1882540975 shutdown_connections 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.654+0000 7f8a6c000640 1 -- 172.21.15.50:0/1882540975 wait complete. 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.655+0000 7f8a6c000640 1 Processor -- start 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.655+0000 7f8a6c000640 1 -- start start 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.656+0000 7f8a6c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a6419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.656+0000 7f8a6c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410dcd0 con 0x7f8a641097d0 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.656+0000 7f8a69600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a6419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.656+0000 7f8a69600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a6419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37552/0 (socket says 172.21.15.50:37552) 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.656+0000 7f8a69600640 1 -- 172.21.15.50:0/1489088847 learned_addr learned my addr 172.21.15.50:0/1489088847 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.657+0000 7f8a69600640 1 -- 172.21.15.50:0/1489088847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6419dcf0 con 0x7f8a641097d0 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.657+0000 7f8a69600640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a6419d7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a580097f0 tx=0x7f8a580327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.657+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a58037040 con 0x7f8a641097d0 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.657+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6419df80 con 0x7f8a641097d0 2024-09-12T03:17:09.286 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.657+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a58030cc0 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.657+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a5803a640 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.658+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641a0c70 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.660+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a64109bd0 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.660+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8a5803b3f0 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.661+0000 7f8a62000640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 0x7f8a4c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.661+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a5806d110 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.661+0000 7f8a68c00640 1 -- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 msgr2=0x7f8a4c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2850856182 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.661+0000 7f8a68c00640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 0x7f8a4c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.665+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a58031e80 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.862+0000 7f8a68c00640 1 -- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 msgr2=0x7f8a4c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2850856182 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.862+0000 7f8a68c00640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 0x7f8a4c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.910+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8a641a1260 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.911+0000 7f8a62000640 1 -- 172.21.15.50:0/1489088847 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f8a5803ba40 con 0x7f8a641097d0 2024-09-12T03:17:09.287 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 msgr2=0x7f8a4c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 0x7f8a4c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 msgr2=0x7f8a6419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a6419d7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a580097f0 tx=0x7f8a580327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 shutdown_connections 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f8a4c038fc0 0x7f8a4c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 --2- 172.21.15.50:0/1489088847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a641097d0 0x7f8a6419d7b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 >> 172.21.15.50:0/1489088847 conn(0x7f8a6407ccf0 msgr2=0x7f8a64107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 shutdown_connections 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:07.914+0000 7f8a6c000640 1 -- 172.21.15.50:0/1489088847 wait complete. 2024-09-12T03:17:09.288 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for the mgr to restart... 2024-09-12T03:17:09.289 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr epoch 9... 2024-09-12T03:17:09.544 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:09.349+0000 7f453de13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:17:09.544 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:09.433+0000 7f453de13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:17:10.664 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:10.388+0000 7f453de13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:17:10.664 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:10.529+0000 7f453de13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:17:10.951 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:10.699+0000 7f453de13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:17:10.951 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:10.773+0000 7f453de13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:17:10.951 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:10.951+0000 7f453de13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:17:11.296 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.035+0000 7f453de13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:17:11.296 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.109+0000 7f453de13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:17:11.296 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.296+0000 7f453de13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:17:11.557 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.376+0000 7f453de13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:17:11.557 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.458+0000 7f453de13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:17:11.558 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.557+0000 7f453de13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:17:11.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.630+0000 7f453de13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:17:11.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:11.703+0000 7f453de13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:17:12.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:12.651+0000 7f453de13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:17:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: Active manager daemon x restarted 2024-09-12T03:17:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: Activating manager daemon x 2024-09-12T03:17:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T03:17:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: mgrmap e10: x(active, starting, since 0.00416683s) 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: Manager daemon x is now available 2024-09-12T03:17:12.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:17:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:17:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:13 smithi050 ceph-mon[30378]: mgrmap e11: x(active, since 1.00772s) 2024-09-12T03:17:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:13 smithi050 ceph-mon[30378]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:17:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:13 smithi050 ceph-mon[30378]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:17:14.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:14.530+0000 7f449b600640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:17:14] ENGINE Error in HTTPServer.serve 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Traceback (most recent call last): 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._connections.run(self.expiration_interval) 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._run(expiration_interval) 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:17:14.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: s = self.context.wrap_socket( 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: return self.sslsocket_class._create( 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self.do_handshake() 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._sslobj.do_handshake() 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:17:14.916 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:17:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:17:14.944 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:17:14.944 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T03:17:14.944 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:17:14.944 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:17:14.944 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.772+0000 7f7490200640 1 Processor -- start 2024-09-12T03:17:14.944 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.772+0000 7f7490200640 1 -- start start 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.772+0000 7f7490200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f7488109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.772+0000 7f7490200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f748810a1a0 con 0x7f74881097d0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.773+0000 7f748d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f7488109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.773+0000 7f748d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f7488109bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40506/0 (socket says 172.21.15.50:40506) 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.773+0000 7f748d800640 1 -- 172.21.15.50:0/336115844 learned_addr learned my addr 172.21.15.50:0/336115844 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.773+0000 7f748d800640 1 -- 172.21.15.50:0/336115844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748810a970 con 0x7f74881097d0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.774+0000 7f748d800640 1 --2- 172.21.15.50:0/336115844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f7488109bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f747c009820 tx=0x7f747c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf30b4a2b086716c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.775+0000 7f7487e00640 1 -- 172.21.15.50:0/336115844 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f747c037070 con 0x7f74881097d0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.775+0000 7f7487e00640 1 -- 172.21.15.50:0/336115844 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c032920 con 0x7f74881097d0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.775+0000 7f7487e00640 1 -- 172.21.15.50:0/336115844 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f747c030810 con 0x7f74881097d0 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 -- 172.21.15.50:0/336115844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 msgr2=0x7f7488109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.945 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 --2- 172.21.15.50:0/336115844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f7488109bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f747c009820 tx=0x7f747c02a270 comp rx=0 tx=0).stop 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 -- 172.21.15.50:0/336115844 shutdown_connections 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 --2- 172.21.15.50:0/336115844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f7488109bd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 -- 172.21.15.50:0/336115844 >> 172.21.15.50:0/336115844 conn(0x7f748807cd20 msgr2=0x7f748807d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 -- 172.21.15.50:0/336115844 shutdown_connections 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.776+0000 7f7490200640 1 -- 172.21.15.50:0/336115844 wait complete. 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f7490200640 1 Processor -- start 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f7490200640 1 -- start start 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f7490200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f748819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f7490200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f748810dd20 con 0x7f74881097d0 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f748d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f748819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f748d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f748819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40516/0 (socket says 172.21.15.50:40516) 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.777+0000 7f748d800640 1 -- 172.21.15.50:0/2291029171 learned_addr learned my addr 172.21.15.50:0/2291029171 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.778+0000 7f748d800640 1 -- 172.21.15.50:0/2291029171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748819dd10 con 0x7f74881097d0 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.778+0000 7f748d800640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f748819d7d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097f0 tx=0x7f747c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.946 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.778+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f747c037040 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.778+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c030cc0 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.778+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f747c03a640 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.778+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748819dfa0 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.779+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a0c90 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.780+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f747c03b3f0 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.780+0000 7f7486000640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.780+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 --> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f747003bbf0 con 0x7f7470039020 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.780+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f747c06c5c0 con 0x7f74881097d0 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.780+0000 7f748ce00640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 msgr2=0x7f747003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2850856182 2024-09-12T03:17:14.947 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.780+0000 7f748ce00640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.981+0000 7f748ce00640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 msgr2=0x7f747003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2850856182 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:09.981+0000 7f748ce00640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:10.382+0000 7f748ce00640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 msgr2=0x7f747003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2850856182 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:10.382+0000 7f748ce00640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:11.183+0000 7f748ce00640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 msgr2=0x7f747003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2850856182 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:11.183+0000 7f748ce00640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:12.655+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f747c06afb0 con 0x7f74881097d0 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:12.655+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 msgr2=0x7f747003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:12.656+0000 7f7486000640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.657+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f747c049ca0 con 0x7f74881097d0 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.657+0000 7f7486000640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f747003cae0 0x7f747003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.657+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f747003bbf0 con 0x7f747003cae0 2024-09-12T03:17:14.948 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.659+0000 7f748ce00640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f747003cae0 0x7f747003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.660+0000 7f748ce00640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f747003cae0 0x7f747003eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7474003980 tx=0x7f74740092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.661+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f747003bbf0 con 0x7f747003cae0 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.669+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7488109bd0 con 0x7f747003cae0 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.669+0000 7f7486000640 1 -- 172.21.15.50:0/2291029171 <== mgr.14152 v2:172.21.15.50:6800/63385930 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7488109bd0 con 0x7f747003cae0 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.669+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f747003cae0 msgr2=0x7f747003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.669+0000 7f7490200640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f747003cae0 0x7f747003eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7474003980 tx=0x7f74740092b0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.669+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 msgr2=0x7f748819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.669+0000 7f7490200640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f748819d7d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097f0 tx=0x7f747c0327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 shutdown_connections 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f747003cae0 0x7f747003eed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:6800/2850856182,v1:172.21.15.50:6801/2850856182] conn(0x7f7470039020 0x7f747003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 --2- 172.21.15.50:0/2291029171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74881097d0 0x7f748819d7d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 >> 172.21.15.50:0/2291029171 conn(0x7f748807cd20 msgr2=0x7f74881074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.950 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 shutdown_connections 2024-09-12T03:17:14.950 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:13.670+0000 7f7490200640 1 -- 172.21.15.50:0/2291029171 wait complete. 2024-09-12T03:17:14.950 INFO:teuthology.orchestra.run.smithi050.stdout:mgr epoch 9 is available 2024-09-12T03:17:14.950 INFO:teuthology.orchestra.run.smithi050.stdout:Generating a dashboard self-signed certificate... 2024-09-12T03:17:15.763 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: [12/Sep/2024:03:17:14] ENGINE Bus STARTING 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: [12/Sep/2024:03:17:14] ENGINE Serving on https://172.21.15.50:7150 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: [12/Sep/2024:03:17:14] ENGINE Error in HTTPServer.serve 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: Traceback (most recent call last): 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._run(expiration_interval) 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:17:15.764 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: s = self.context.wrap_socket( 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: return self.sslsocket_class._create( 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: self.do_handshake() 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._sslobj.do_handshake() 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: [12/Sep/2024:03:17:14] ENGINE Serving on http://172.21.15.50:8765 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: [12/Sep/2024:03:17:14] ENGINE Bus STARTED 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:15.765 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:15 smithi050 ceph-mon[30378]: mgrmap e12: x(active, since 2s) 2024-09-12T03:17:16.402 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.459+0000 7fbca6e00640 1 Processor -- start 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.459+0000 7fbca6e00640 1 -- start start 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.460+0000 7fbca6e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca0109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.460+0000 7fbca6e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010a1e0 con 0x7fbca0109810 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.460+0000 7fbc9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca0109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.460+0000 7fbc9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca0109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40578/0 (socket says 172.21.15.50:40578) 2024-09-12T03:17:16.407 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.460+0000 7fbc9fe00640 1 -- 172.21.15.50:0/1149242407 learned_addr learned my addr 172.21.15.50:0/1149242407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.461+0000 7fbc9fe00640 1 -- 172.21.15.50:0/1149242407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca010a950 con 0x7fbca0109810 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.462+0000 7fbc9fe00640 1 --2- 172.21.15.50:0/1149242407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca0109c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009820 tx=0x7fbc9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f6ba89ceca0851 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbc9ea00640 1 -- 172.21.15.50:0/1149242407 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc90037070 con 0x7fbca0109810 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbc9ea00640 1 -- 172.21.15.50:0/1149242407 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc90032920 con 0x7fbca0109810 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbc9ea00640 1 -- 172.21.15.50:0/1149242407 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc90030810 con 0x7fbca0109810 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbca6e00640 1 -- 172.21.15.50:0/1149242407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 msgr2=0x7fbca0109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbca6e00640 1 --2- 172.21.15.50:0/1149242407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca0109c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009820 tx=0x7fbc9002a270 comp rx=0 tx=0).stop 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbca6e00640 1 -- 172.21.15.50:0/1149242407 shutdown_connections 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbca6e00640 1 --2- 172.21.15.50:0/1149242407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca0109c10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbca6e00640 1 -- 172.21.15.50:0/1149242407 >> 172.21.15.50:0/1149242407 conn(0x7fbca007ccf0 msgr2=0x7fbca007d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.463+0000 7fbca6e00640 1 -- 172.21.15.50:0/1149242407 shutdown_connections 2024-09-12T03:17:16.408 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.464+0000 7fbca6e00640 1 -- 172.21.15.50:0/1149242407 wait complete. 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.464+0000 7fbca6e00640 1 Processor -- start 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbca6e00640 1 -- start start 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbca6e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbca6e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010dd00 con 0x7fbca0109810 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbc9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbc9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40590/0 (socket says 172.21.15.50:40590) 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbc9fe00640 1 -- 172.21.15.50:0/2075820905 learned_addr learned my addr 172.21.15.50:0/2075820905 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.465+0000 7fbc9fe00640 1 -- 172.21.15.50:0/2075820905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca019dcf0 con 0x7fbca0109810 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.466+0000 7fbc9fe00640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca019d7b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc900097f0 tx=0x7fbc900327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.466+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc90037040 con 0x7fbca0109810 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.466+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc90030c40 con 0x7fbca0109810 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.466+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc9003a5d0 con 0x7fbca0109810 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.466+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca019df80 con 0x7fbca0109810 2024-09-12T03:17:16.409 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.466+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a0c70 con 0x7fbca0109810 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.467+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fbc9003b3f0 con 0x7fbca0109810 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.467+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca0109c90 con 0x7fbca0109810 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.468+0000 7fbc9cc00640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbc84038fc0 0x7fbc8403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.468+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc9006c630 con 0x7fbca0109810 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.468+0000 7fbc9f400640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbc84038fc0 0x7fbc8403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.469+0000 7fbc9f400640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbc84038fc0 0x7fbc8403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098c0 tx=0x7fbc8c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.473+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc90040210 con 0x7fbca0109810 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.664+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbca0107c70 con 0x7fbc84038fc0 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.763+0000 7fbc9cc00640 1 -- 172.21.15.50:0/2075820905 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbca0107c70 con 0x7fbc84038fc0 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.765+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbc84038fc0 msgr2=0x7fbc8403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.765+0000 7fbca6e00640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbc84038fc0 0x7fbc8403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098c0 tx=0x7fbc8c006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.765+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 msgr2=0x7fbca019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.765+0000 7fbca6e00640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca019d7b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc900097f0 tx=0x7fbc900327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.410 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.766+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 shutdown_connections 2024-09-12T03:17:16.411 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.766+0000 7fbca6e00640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbc84038fc0 0x7fbc8403b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.411 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.766+0000 7fbca6e00640 1 --2- 172.21.15.50:0/2075820905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbca0109810 0x7fbca019d7b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.411 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.766+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 >> 172.21.15.50:0/2075820905 conn(0x7fbca007ccf0 msgr2=0x7fbca0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.411 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.766+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 shutdown_connections 2024-09-12T03:17:16.411 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:15.766+0000 7fbca6e00640 1 -- 172.21.15.50:0/2075820905 wait complete. 2024-09-12T03:17:16.411 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial admin user... 2024-09-12T03:17:17.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:16 smithi050 ceph-mon[30378]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:17.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:16 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:17.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:16 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:17.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:16 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:18.252 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kwtlFsVbIVCDyLVBfGHpa.6f/RdDu3PiAS/JXyJ1vbecpFs6DMpfi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726111037, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T03:17:18.252 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.896+0000 7f1c1d800640 1 Processor -- start 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.896+0000 7f1c1d800640 1 -- start start 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.897+0000 7f1c1d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c18107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.897+0000 7f1c1d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18107f00 con 0x7f1c18107530 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.897+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c18107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.897+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c18107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40594/0 (socket says 172.21.15.50:40594) 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.897+0000 7f1c16a00640 1 -- 172.21.15.50:0/1823468628 learned_addr learned my addr 172.21.15.50:0/1823468628 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.897+0000 7f1c16a00640 1 -- 172.21.15.50:0/1823468628 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18108730 con 0x7f1c18107530 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.898+0000 7f1c16a00640 1 --2- 172.21.15.50:0/1823468628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c18107930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009820 tx=0x7f1c0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fb76d6d482706d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c16000640 1 -- 172.21.15.50:0/1823468628 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08037070 con 0x7f1c18107530 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c16000640 1 -- 172.21.15.50:0/1823468628 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c08032920 con 0x7f1c18107530 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c16000640 1 -- 172.21.15.50:0/1823468628 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08030810 con 0x7f1c18107530 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c1d800640 1 -- 172.21.15.50:0/1823468628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 msgr2=0x7f1c18107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c1d800640 1 --2- 172.21.15.50:0/1823468628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c18107930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009820 tx=0x7f1c0802a270 comp rx=0 tx=0).stop 2024-09-12T03:17:18.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c1d800640 1 -- 172.21.15.50:0/1823468628 shutdown_connections 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c1d800640 1 --2- 172.21.15.50:0/1823468628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c18107930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c1d800640 1 -- 172.21.15.50:0/1823468628 >> 172.21.15.50:0/1823468628 conn(0x7f1c18102cc0 msgr2=0x7f1c18105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.899+0000 7f1c1d800640 1 -- 172.21.15.50:0/1823468628 shutdown_connections 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.900+0000 7f1c1d800640 1 -- 172.21.15.50:0/1823468628 wait complete. 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.900+0000 7f1c1d800640 1 Processor -- start 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c1d800640 1 -- start start 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c1d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c1819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c1d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810bae0 con 0x7f1c18107530 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c1819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c1819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40608/0 (socket says 172.21.15.50:40608) 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c16a00640 1 -- 172.21.15.50:0/1420054802 learned_addr learned my addr 172.21.15.50:0/1420054802 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.901+0000 7f1c16a00640 1 -- 172.21.15.50:0/1420054802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1819dcf0 con 0x7f1c18107530 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.902+0000 7f1c16a00640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c1819d7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1c080097f0 tx=0x7f1c080327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.254 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.902+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c08037040 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.902+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c08030cc0 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.902+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1819df80 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.902+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c0803a640 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.902+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a0c70 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.903+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1c0803b3f0 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.903+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1810bc60 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.904+0000 7f1c0f400640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1bfc039020 0x7f1bfc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.904+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c0806c6a0 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.904+0000 7f1c0fe00640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1bfc039020 0x7f1bfc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.905+0000 7f1c0fe00640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1bfc039020 0x7f1bfc03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098c0 tx=0x7f1c00006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:16.909+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c080328c0 con 0x7f1c18107530 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.120+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1c181a0e40 con 0x7f1bfc039020 2024-09-12T03:17:18.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.371+0000 7f1c0f400640 1 -- 172.21.15.50:0/1420054802 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1c181a0e40 con 0x7f1bfc039020 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.373+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1bfc039020 msgr2=0x7f1bfc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.373+0000 7f1c1d800640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1bfc039020 0x7f1bfc03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098c0 tx=0x7f1c00006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.373+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 msgr2=0x7f1c1819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.373+0000 7f1c1d800640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c1819d7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1c080097f0 tx=0x7f1c080327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.374+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 shutdown_connections 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.374+0000 7f1c1d800640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1bfc039020 0x7f1bfc03b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.374+0000 7f1c1d800640 1 --2- 172.21.15.50:0/1420054802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c18107530 0x7f1c1819d7b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.374+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 >> 172.21.15.50:0/1420054802 conn(0x7f1c18102cc0 msgr2=0x7f1c18103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.374+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 shutdown_connections 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:17.374+0000 7f1c1d800640 1 -- 172.21.15.50:0/1420054802 wait complete. 2024-09-12T03:17:18.256 INFO:teuthology.orchestra.run.smithi050.stdout:Fetching dashboard port number... 2024-09-12T03:17:18.568 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:18 smithi050 ceph-mon[30378]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:18.569 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:19.484 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:19 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4087663158' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.034+0000 7f5d52400640 1 Processor -- start 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.034+0000 7f5d52400640 1 -- start start 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.034+0000 7f5d52400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c07c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.034+0000 7f5d52400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c07c7a0 con 0x7f5d4c07de00 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.035+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c07c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.035+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c07c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52896/0 (socket says 172.21.15.50:52896) 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.035+0000 7f5d4b400640 1 -- 172.21.15.50:0/2474659387 learned_addr learned my addr 172.21.15.50:0/2474659387 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.035+0000 7f5d4b400640 1 -- 172.21.15.50:0/2474659387 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c07c920 con 0x7f5d4c07de00 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.036+0000 7f5d4b400640 1 --2- 172.21.15.50:0/2474659387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c07c260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009820 tx=0x7f5d3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfaae181d8d9d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d4a000640 1 -- 172.21.15.50:0/2474659387 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c037070 con 0x7f5d4c07de00 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d4a000640 1 -- 172.21.15.50:0/2474659387 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c032920 con 0x7f5d4c07de00 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d4a000640 1 -- 172.21.15.50:0/2474659387 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c030810 con 0x7f5d4c07de00 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 -- 172.21.15.50:0/2474659387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 msgr2=0x7f5d4c07c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 --2- 172.21.15.50:0/2474659387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c07c260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009820 tx=0x7f5d3c02a270 comp rx=0 tx=0).stop 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 -- 172.21.15.50:0/2474659387 shutdown_connections 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 --2- 172.21.15.50:0/2474659387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c07c260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 -- 172.21.15.50:0/2474659387 >> 172.21.15.50:0/2474659387 conn(0x7f5d4c102cc0 msgr2=0x7f5d4c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 -- 172.21.15.50:0/2474659387 shutdown_connections 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.037+0000 7f5d52400640 1 -- 172.21.15.50:0/2474659387 wait complete. 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.038+0000 7f5d52400640 1 Processor -- start 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.038+0000 7f5d52400640 1 -- start start 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d52400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d52400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10db80 con 0x7f5d4c07de00 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c19d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52902/0 (socket says 172.21.15.50:52902) 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d4b400640 1 -- 172.21.15.50:0/4087663158 learned_addr learned my addr 172.21.15.50:0/4087663158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d4b400640 1 -- 172.21.15.50:0/4087663158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c19dc90 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.039+0000 7f5d4b400640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c19d750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c030380 tx=0x7f5d3c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.040+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c037040 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.040+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c040070 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.040+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c19df20 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.040+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c03a7b0 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.040+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1a0c10 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.041+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5d3c03b420 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.041+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c07c2e0 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.041+0000 7f5d43e00640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5d30038fc0 0x7f5d3003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.042+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d3c06c800 con 0x7f5d4c07de00 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.042+0000 7f5d4aa00640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5d30038fc0 0x7f5d3003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.042+0000 7f5d4aa00640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5d30038fc0 0x7f5d3003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5d380098c0 tx=0x7f5d38006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.047+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d3c043360 con 0x7f5d4c07de00 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.232+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5d4c1a1200 con 0x7f5d4c07de00 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.233+0000 7f5d43e00640 1 -- 172.21.15.50:0/4087663158 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5d3c040220 con 0x7f5d4c07de00 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5d30038fc0 msgr2=0x7f5d3003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5d30038fc0 0x7f5d3003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5d380098c0 tx=0x7f5d38006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 msgr2=0x7f5d4c19d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c19d750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c030380 tx=0x7f5d3c032670 comp rx=0 tx=0).stop 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 shutdown_connections 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5d30038fc0 0x7f5d3003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 --2- 172.21.15.50:0/4087663158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d4c07de00 0x7f5d4c19d750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 >> 172.21.15.50:0/4087663158 conn(0x7f5d4c102cc0 msgr2=0x7f5d4c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 shutdown_connections 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:19.235+0000 7f5d52400640 1 -- 172.21.15.50:0/4087663158 wait complete. 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi050.stdout:Ceph Dashboard is now available at: 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi050.stdout: URL: https://smithi050.front.sepia.ceph.com:8443/ 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi050.stdout: User: admin 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi050.stdout: Password: cd7mpcfbfm 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi050.stdout:Saving cluster configuration to /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config directory 2024-09-12T03:17:20.565 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:20 smithi050 ceph-mon[30378]: mgrmap e13: x(active, since 6s) 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.273+0000 7fc0c1200640 1 Processor -- start 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.274+0000 7fc0c1200640 1 -- start start 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.274+0000 7fc0c1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.274+0000 7fc0c1200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc107f00 con 0x7fc0bc107530 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.274+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.274+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52914/0 (socket says 172.21.15.50:52914) 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.274+0000 7fc0b9e00640 1 -- 172.21.15.50:0/641148393 learned_addr learned my addr 172.21.15.50:0/641148393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.275+0000 7fc0b9e00640 1 -- 172.21.15.50:0/641148393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc108730 con 0x7fc0bc107530 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.276+0000 7fc0b9e00640 1 --2- 172.21.15.50:0/641148393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc107930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009820 tx=0x7fc0ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a917bb78073872 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.276+0000 7fc0b9400640 1 -- 172.21.15.50:0/641148393 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0ac037070 con 0x7fc0bc107530 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.276+0000 7fc0b9400640 1 -- 172.21.15.50:0/641148393 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac032920 con 0x7fc0bc107530 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.276+0000 7fc0b9400640 1 -- 172.21.15.50:0/641148393 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0ac030810 con 0x7fc0bc107530 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.276+0000 7fc0c1200640 1 -- 172.21.15.50:0/641148393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 msgr2=0x7fc0bc107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.277+0000 7fc0c1200640 1 --2- 172.21.15.50:0/641148393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc107930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009820 tx=0x7fc0ac02a270 comp rx=0 tx=0).stop 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.277+0000 7fc0c1200640 1 -- 172.21.15.50:0/641148393 shutdown_connections 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.277+0000 7fc0c1200640 1 --2- 172.21.15.50:0/641148393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc107930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.277+0000 7fc0c1200640 1 -- 172.21.15.50:0/641148393 >> 172.21.15.50:0/641148393 conn(0x7fc0bc102cc0 msgr2=0x7fc0bc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.277+0000 7fc0c1200640 1 -- 172.21.15.50:0/641148393 shutdown_connections 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.277+0000 7fc0c1200640 1 -- 172.21.15.50:0/641148393 wait complete. 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0c1200640 1 Processor -- start 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0c1200640 1 -- start start 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0c1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0c1200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10bae0 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52918/0 (socket says 172.21.15.50:52918) 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.278+0000 7fc0b9e00640 1 -- 172.21.15.50:0/525539937 learned_addr learned my addr 172.21.15.50:0/525539937 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0b9e00640 1 -- 172.21.15.50:0/525539937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc19dd10 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0b9e00640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc19d7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0097f0 tx=0x7fc0ac0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0ac037040 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac030cc0 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0ac03a640 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc19dfa0 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.279+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a0c90 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.280+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc0ac03b3f0 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.281+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc10bc60 con 0x7fc0bc107530 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.281+0000 7fc0b3400640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc0a0039060 0x7fc0a003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.281+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc0ac06c7a0 con 0x7fc0bc107530 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.281+0000 7fc0b1400640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc0a0039060 0x7fc0a003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.282+0000 7fc0b1400640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc0a0039060 0x7fc0a003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40098c0 tx=0x7fc0a4006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.286+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0ac043e10 con 0x7fc0bc107530 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.565+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc0bc1a0e60 con 0x7fc0bc107530 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.568+0000 7fc0b3400640 1 -- 172.21.15.50:0/525539937 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc0ac03ba40 con 0x7fc0bc107530 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc0a0039060 msgr2=0x7fc0a003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc0a0039060 0x7fc0a003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40098c0 tx=0x7fc0a4006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 msgr2=0x7fc0bc19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc19d7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0097f0 tx=0x7fc0ac0327b0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 shutdown_connections 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc0a0039060 0x7fc0a003b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 --2- 172.21.15.50:0/525539937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0bc107530 0x7fc0bc19d7d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 >> 172.21.15.50:0/525539937 conn(0x7fc0bc102cc0 msgr2=0x7fc0bc103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 shutdown_connections 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-09-12T03:17:20.570+0000 7fc0c1200640 1 -- 172.21.15.50:0/525539937 wait complete. 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.129 INFO:teuthology.orchestra.run.smithi050.stdout: ceph telemetry on 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi050.stdout:For more information see: 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi050.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi050.stdout:Bootstrap complete. 2024-09-12T03:17:21.182 INFO:tasks.cephadm:Fetching config... 2024-09-12T03:17:21.182 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:17:21.182 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T03:17:21.218 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T03:17:21.219 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:17:21.219 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T03:17:21.294 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T03:17:21.294 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:17:21.294 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T03:17:21.391 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T03:17:21.391 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:17:21.392 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T03:17:21.467 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T03:17:21.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:17:21.591 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:21.618 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:17:21.661 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:21.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:21 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/525539937' entity='client.admin' 2024-09-12T03:17:21.673 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:17:21.714 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcr2GovWfTTkoHSjdYcKGHZPHqNj6xQnH3zfJaHVhQ0wYKllsBREtSj73tmQHJbgW08d5FmHDx22GeCK1u78ZFaQ86X8XWhwQCYWC5MX2i43PEPRkEQo4NGwt6t5pzTwc93E35TWIz3bCh4JZEvC00NIoON03/FH9ZrXMcqLekWJjFidxU+C+2bmrWL+rnf2GUd9/PQN8ByQT8r2hOejg47QQLwPtW9B9HQbdfzBgpHeHwQO8lzdBrtYRON/06tcbOuMHXzTt3ClDZNek/M0osY7uETrJ8YfoaKvbMC2ial/VINOxNlquJiXsT34C9jO0NSkWe6p5ElY/6eE6qZexD+ONfVzrWKBeHmv0d17HScdcral/KNGZzEI9V6SCe7alNY+sQi5A22cD57KUGRvK8yqqSxs53JI3D4S3//PH94fYTtSDmUij5eE4hfz06s8fuG558EZ7nzb62gerqtZbFoTK8jwBiEnjXP72eKqlv3IA/iIRSJJhgnjfPE58S/RM= ceph-389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:21.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T03:17:22.103 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 -- 172.21.15.50:0/523693797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 msgr2=0x7f98a0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 --2- 172.21.15.50:0/523693797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0074620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9890009880 tx=0x7f989002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 -- 172.21.15.50:0/523693797 shutdown_connections 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 --2- 172.21.15.50:0/523693797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0074620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 -- 172.21.15.50:0/523693797 >> 172.21.15.50:0/523693797 conn(0x7f98a00fcbd0 msgr2=0x7f98a00feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 -- 172.21.15.50:0/523693797 shutdown_connections 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.616+0000 7f989f200640 1 -- 172.21.15.50:0/523693797 wait complete. 2024-09-12T03:17:22.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.617+0000 7f989f200640 1 Processor -- start 2024-09-12T03:17:22.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.618+0000 7f989f200640 1 -- start start 2024-09-12T03:17:22.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.618+0000 7f989f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.618+0000 7f989f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0109720 con 0x7f98a00761c0 2024-09-12T03:17:22.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.618+0000 7f989de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.618+0000 7f989de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52928/0 (socket says 172.21.15.50:52928) 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.618+0000 7f989de00640 1 -- 172.21.15.50:0/1364190750 learned_addr learned my addr 172.21.15.50:0/1364190750 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.619+0000 7f989de00640 1 -- 172.21.15.50:0/1364190750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a01992a0 con 0x7f98a00761c0 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.619+0000 7f989de00640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0198d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9890004f60 tx=0x7f98900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.619+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f989003f070 con 0x7f98a00761c0 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.619+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9890003ad0 con 0x7f98a00761c0 2024-09-12T03:17:22.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.619+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9890035030 con 0x7f98a00761c0 2024-09-12T03:17:22.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.619+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a0199530 con 0x7f98a00761c0 2024-09-12T03:17:22.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.620+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a019c220 con 0x7f98a00761c0 2024-09-12T03:17:22.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.621+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a0109ea0 con 0x7f98a00761c0 2024-09-12T03:17:22.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.621+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9890042070 con 0x7f98a00761c0 2024-09-12T03:17:22.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.621+0000 7f9896a00640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9884039060 0x7f988403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.622+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f989006cf70 con 0x7f98a00761c0 2024-09-12T03:17:22.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.622+0000 7f989d400640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9884039060 0x7f988403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.623+0000 7f989d400640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9884039060 0x7f988403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098c0 tx=0x7f988c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.629+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f989003a3c0 con 0x7f98a00761c0 2024-09-12T03:17:22.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.823+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f98a002d7c0 con 0x7f98a00761c0 2024-09-12T03:17:22.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.825+0000 7f9896a00640 1 -- 172.21.15.50:0/1364190750 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f98900412c0 con 0x7f98a00761c0 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9884039060 msgr2=0x7f988403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9884039060 0x7f988403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098c0 tx=0x7f988c006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 msgr2=0x7f98a0198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0198d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9890004f60 tx=0x7f98900048f0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 shutdown_connections 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9884039060 0x7f988403b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 --2- 172.21.15.50:0/1364190750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a00761c0 0x7f98a0198d60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 >> 172.21.15.50:0/1364190750 conn(0x7f98a00fcbd0 msgr2=0x7f98a00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 shutdown_connections 2024-09-12T03:17:22.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:22.828+0000 7f989f200640 1 -- 172.21.15.50:0/1364190750 wait complete. 2024-09-12T03:17:23.681 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T03:17:23.681 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T03:17:23.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:23 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1364190750' entity='client.admin' 2024-09-12T03:17:24.126 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 -- 172.21.15.50:0/2466756149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 msgr2=0x7f9f00074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 --2- 172.21.15.50:0/2466756149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00074620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009880 tx=0x7f9ef002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 -- 172.21.15.50:0/2466756149 shutdown_connections 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 --2- 172.21.15.50:0/2466756149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00074620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 -- 172.21.15.50:0/2466756149 >> 172.21.15.50:0/2466756149 conn(0x7f9f000fcbd0 msgr2=0x7f9f000feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 -- 172.21.15.50:0/2466756149 shutdown_connections 2024-09-12T03:17:24.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.878+0000 7f9effe00640 1 -- 172.21.15.50:0/2466756149 wait complete. 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9effe00640 1 Processor -- start 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9effe00640 1 -- start start 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9effe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9effe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00109720 con 0x7f9f000761c0 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9efea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9efea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52944/0 (socket says 172.21.15.50:52944) 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.879+0000 7f9efea00640 1 -- 172.21.15.50:0/3457606245 learned_addr learned my addr 172.21.15.50:0/3457606245 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:24.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.880+0000 7f9efea00640 1 -- 172.21.15.50:0/3457606245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f001992f0 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.880+0000 7f9efea00640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00198db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0004f60 tx=0x7f9ef00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.880+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ef003f070 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.880+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef0003ad0 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.880+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f00199580 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.880+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ef0035030 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.881+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0019c230 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.881+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f00109ea0 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.882+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9ef0042070 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.882+0000 7f9ef7e00640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9ee4039060 0x7f9ee403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.882+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ef006cf20 con 0x7f9f000761c0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.882+0000 7f9ef6000640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9ee4039060 0x7f9ee403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.883+0000 7f9ef6000640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9ee4039060 0x7f9ee403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80098c0 tx=0x7f9ee8006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:24.887+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ef003a3c0 con 0x7f9f000761c0 2024-09-12T03:17:25.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.082+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9f0002d940 con 0x7f9ee4039060 2024-09-12T03:17:25.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.086+0000 7f9ef7e00640 1 -- 172.21.15.50:0/3457606245 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f0002d940 con 0x7f9ee4039060 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.089+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9ee4039060 msgr2=0x7f9ee403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.089+0000 7f9effe00640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9ee4039060 0x7f9ee403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80098c0 tx=0x7f9ee8006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.089+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 msgr2=0x7f9f00198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.089+0000 7f9effe00640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00198db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0004f60 tx=0x7f9ef00048f0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.090+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 shutdown_connections 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.090+0000 7f9effe00640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9ee4039060 0x7f9ee403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.090+0000 7f9effe00640 1 --2- 172.21.15.50:0/3457606245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f000761c0 0x7f9f00198db0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.090+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 >> 172.21.15.50:0/3457606245 conn(0x7f9f000fcbd0 msgr2=0x7f9f000fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.090+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 shutdown_connections 2024-09-12T03:17:25.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:25.090+0000 7f9effe00640 1 -- 172.21.15.50:0/3457606245 wait complete. 2024-09-12T03:17:25.731 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi082 2024-09-12T03:17:25.731 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:17:25.732 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:17:25.750 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:17:25.751 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:25.808 INFO:tasks.cephadm:Adding host smithi082 to orchestrator... 2024-09-12T03:17:25.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch host add smithi082 2024-09-12T03:17:26.385 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:26 smithi050 ceph-mon[30378]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:26.385 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:26 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.998+0000 7f4c2d400640 1 -- 172.21.15.50:0/949513425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 msgr2=0x7f4c28074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.998+0000 7f4c2d400640 1 --2- 172.21.15.50:0/949513425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c28074680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009880 tx=0x7f4c1802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.998+0000 7f4c2d400640 1 -- 172.21.15.50:0/949513425 shutdown_connections 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.998+0000 7f4c2d400640 1 --2- 172.21.15.50:0/949513425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c28074680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.998+0000 7f4c2d400640 1 -- 172.21.15.50:0/949513425 >> 172.21.15.50:0/949513425 conn(0x7f4c280fcb90 msgr2=0x7f4c280fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.998+0000 7f4c2d400640 1 -- 172.21.15.50:0/949513425 shutdown_connections 2024-09-12T03:17:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.999+0000 7f4c2d400640 1 -- 172.21.15.50:0/949513425 wait complete. 2024-09-12T03:17:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:26.999+0000 7f4c2d400640 1 Processor -- start 2024-09-12T03:17:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.000+0000 7f4c2d400640 1 -- start start 2024-09-12T03:17:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.000+0000 7f4c2d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c2819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.000+0000 7f4c2d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810db60 con 0x7f4c28076220 2024-09-12T03:17:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.000+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c2819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.000+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c2819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52962/0 (socket says 172.21.15.50:52962) 2024-09-12T03:17:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.000+0000 7f4c27e00640 1 -- 172.21.15.50:0/2440394551 learned_addr learned my addr 172.21.15.50:0/2440394551 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.001+0000 7f4c27e00640 1 -- 172.21.15.50:0/2440394551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2819d710 con 0x7f4c28076220 2024-09-12T03:17:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.001+0000 7f4c27e00640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c2819d1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c18004f60 tx=0x7f4c180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.001+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c1803f070 con 0x7f4c28076220 2024-09-12T03:17:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.001+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c18004cc0 con 0x7f4c28076220 2024-09-12T03:17:27.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.001+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2819d9a0 con 0x7f4c28076220 2024-09-12T03:17:27.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.002+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c281a0690 con 0x7f4c28076220 2024-09-12T03:17:27.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.002+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18035030 con 0x7f4c28076220 2024-09-12T03:17:27.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.003+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2810e2a0 con 0x7f4c28076220 2024-09-12T03:17:27.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.003+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4c18042070 con 0x7f4c28076220 2024-09-12T03:17:27.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.004+0000 7f4c24c00640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f4c0c0390c0 0x7f4c0c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.004+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4c1806cdd0 con 0x7f4c28076220 2024-09-12T03:17:27.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.004+0000 7f4c27400640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f4c0c0390c0 0x7f4c0c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.005+0000 7f4c27400640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f4c0c0390c0 0x7f4c0c03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098c0 tx=0x7f4c14006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.009+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c18036df0 con 0x7f4c28076220 2024-09-12T03:17:27.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:27.204+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}) -- 0x7f4c2802d830 con 0x7f4c0c0390c0 2024-09-12T03:17:27.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:27.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:27.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:27.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:27.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:27.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:27 smithi050 ceph-mon[30378]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-12T03:17:28.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:28 smithi050 ceph-mon[30378]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:28.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:28 smithi050 ceph-mon[30378]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:28.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:28 smithi050 ceph-mon[30378]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:28.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:28.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:28.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:29.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:29 smithi050 ceph-mon[30378]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:30.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:29 smithi050 ceph-mon[30378]: Deploying cephadm binary to smithi082 2024-09-12T03:17:30.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:30.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:30.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:30.179 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:31.179 INFO:teuthology.orchestra.run.smithi050.stdout:Added host 'smithi082' with addr '172.21.15.82' 2024-09-12T03:17:31.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.178+0000 7f4c24c00640 1 -- 172.21.15.50:0/2440394551 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4c2802d830 con 0x7f4c0c0390c0 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.181+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f4c0c0390c0 msgr2=0x7f4c0c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.181+0000 7f4c2d400640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f4c0c0390c0 0x7f4c0c03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098c0 tx=0x7f4c14006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.181+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 msgr2=0x7f4c2819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.181+0000 7f4c2d400640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c2819d1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c18004f60 tx=0x7f4c180048f0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.182+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 shutdown_connections 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.182+0000 7f4c2d400640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f4c0c0390c0 0x7f4c0c03b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.182+0000 7f4c2d400640 1 --2- 172.21.15.50:0/2440394551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c28076220 0x7f4c2819d1d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.182+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 >> 172.21.15.50:0/2440394551 conn(0x7f4c280fcb90 msgr2=0x7f4c280fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.182+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 shutdown_connections 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:31.182+0000 7f4c2d400640 1 -- 172.21.15.50:0/2440394551 wait complete. 2024-09-12T03:17:31.793 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:17:32.156 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:32.439 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:32.439 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:32 smithi050 ceph-mon[30378]: Added host smithi082 2024-09-12T03:17:32.439 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:32.439 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:32.440 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.666+0000 7fba61000640 1 -- 172.21.15.50:0/3621598645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 msgr2=0x7fba5c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.666+0000 7fba61000640 1 --2- 172.21.15.50:0/3621598645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c1019e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009880 tx=0x7fba4c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.667+0000 7fba61000640 1 -- 172.21.15.50:0/3621598645 shutdown_connections 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.667+0000 7fba61000640 1 --2- 172.21.15.50:0/3621598645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c1019e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.667+0000 7fba61000640 1 -- 172.21.15.50:0/3621598645 >> 172.21.15.50:0/3621598645 conn(0x7fba5c0fd0a0 msgr2=0x7fba5c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.667+0000 7fba61000640 1 -- 172.21.15.50:0/3621598645 shutdown_connections 2024-09-12T03:17:32.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.667+0000 7fba61000640 1 -- 172.21.15.50:0/3621598645 wait complete. 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba61000640 1 Processor -- start 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba61000640 1 -- start start 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba61000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba61000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c10ba50 con 0x7fba5c101600 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba5b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba5b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45528/0 (socket says 172.21.15.50:45528) 2024-09-12T03:17:32.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.668+0000 7fba5b400640 1 -- 172.21.15.50:0/1465101251 learned_addr learned my addr 172.21.15.50:0/1465101251 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:32.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.669+0000 7fba5b400640 1 -- 172.21.15.50:0/1465101251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1996a0 con 0x7fba5c101600 2024-09-12T03:17:32.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.669+0000 7fba5b400640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c199160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009230 tx=0x7fba4c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.669+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba4c03f070 con 0x7fba5c101600 2024-09-12T03:17:32.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.669+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c03bc80 con 0x7fba5c101600 2024-09-12T03:17:32.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.669+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba4c035030 con 0x7fba5c101600 2024-09-12T03:17:32.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.669+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c199990 con 0x7fba5c101600 2024-09-12T03:17:32.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.670+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c19da30 con 0x7fba5c101600 2024-09-12T03:17:32.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.671+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fba4c042070 con 0x7fba5c101600 2024-09-12T03:17:32.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.671+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba5c102ae0 con 0x7fba5c101600 2024-09-12T03:17:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.671+0000 7fba58c00640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fba400390c0 0x7fba4003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.671+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fba4c06c810 con 0x7fba5c101600 2024-09-12T03:17:32.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.671+0000 7fba53e00640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fba400390c0 0x7fba4003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.672+0000 7fba53e00640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fba400390c0 0x7fba4003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba440098c0 tx=0x7fba44006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.676+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba4c042360 con 0x7fba5c101600 2024-09-12T03:17:32.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.867+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba5c101a90 con 0x7fba400390c0 2024-09-12T03:17:32.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.868+0000 7fba58c00640 1 -- 172.21.15.50:0/1465101251 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fba5c101a90 con 0x7fba400390c0 2024-09-12T03:17:32.869 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:32.869 INFO:teuthology.orchestra.run.smithi050.stdout:[{"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}] 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fba400390c0 msgr2=0x7fba4003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fba400390c0 0x7fba4003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba440098c0 tx=0x7fba44006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 msgr2=0x7fba5c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c199160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009230 tx=0x7fba4c003d40 comp rx=0 tx=0).stop 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 shutdown_connections 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fba400390c0 0x7fba4003b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 --2- 172.21.15.50:0/1465101251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba5c101600 0x7fba5c199160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 >> 172.21.15.50:0/1465101251 conn(0x7fba5c0fd0a0 msgr2=0x7fba5c1094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 shutdown_connections 2024-09-12T03:17:32.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:32.871+0000 7fba61000640 1 -- 172.21.15.50:0/1465101251 wait complete. 2024-09-12T03:17:33.530 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-09-12T03:17:33.531 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:17:33.531 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:17:33.547 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:17:33.547 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:33.603 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-09-12T03:17:33.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch host add smithi142 2024-09-12T03:17:33.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:33 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:33.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:33 smithi050 ceph-mon[30378]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:33.969 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.465+0000 7f826e600640 1 -- 172.21.15.50:0/2259895057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 msgr2=0x7f8268074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.465+0000 7f826e600640 1 --2- 172.21.15.50:0/2259895057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f8268074970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8258009880 tx=0x7f825802a670 comp rx=0 tx=0).stop 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.465+0000 7f826e600640 1 -- 172.21.15.50:0/2259895057 shutdown_connections 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.465+0000 7f826e600640 1 --2- 172.21.15.50:0/2259895057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f8268074970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.465+0000 7f826e600640 1 -- 172.21.15.50:0/2259895057 >> 172.21.15.50:0/2259895057 conn(0x7f82680fd230 msgr2=0x7f82680ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.466+0000 7f826e600640 1 -- 172.21.15.50:0/2259895057 shutdown_connections 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.466+0000 7f826e600640 1 -- 172.21.15.50:0/2259895057 wait complete. 2024-09-12T03:17:34.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.466+0000 7f826e600640 1 Processor -- start 2024-09-12T03:17:34.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.467+0000 7f826e600640 1 -- start start 2024-09-12T03:17:34.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.467+0000 7f826e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f82681a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.467+0000 7f826e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8268112290 con 0x7f8268074590 2024-09-12T03:17:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.467+0000 7f8267400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f82681a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.467+0000 7f8267400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f82681a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45548/0 (socket says 172.21.15.50:45548) 2024-09-12T03:17:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.467+0000 7f8267400640 1 -- 172.21.15.50:0/2347142667 learned_addr learned my addr 172.21.15.50:0/2347142667 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:34.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f8267400640 1 -- 172.21.15.50:0/2347142667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681a3910 con 0x7f8268074590 2024-09-12T03:17:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f8267400640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f82681a33d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8258009230 tx=0x7f8258003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825803f070 con 0x7f8268074590 2024-09-12T03:17:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f825803bc80 con 0x7f8268074590 2024-09-12T03:17:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8258035030 con 0x7f8268074590 2024-09-12T03:17:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681a3ba0 con 0x7f8268074590 2024-09-12T03:17:34.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.468+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826819d590 con 0x7f8268074590 2024-09-12T03:17:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.469+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8258042070 con 0x7f8268074590 2024-09-12T03:17:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.469+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8268075cd0 con 0x7f8268074590 2024-09-12T03:17:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.470+0000 7f825fe00640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f824c0390c0 0x7f824c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.470+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f825806cd80 con 0x7f8268074590 2024-09-12T03:17:34.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.470+0000 7f8266a00640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f824c0390c0 0x7f824c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.471+0000 7f8266a00640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f824c0390c0 0x7f824c03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f82540098c0 tx=0x7f8254006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.475+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82580372d0 con 0x7f8268074590 2024-09-12T03:17:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:34 smithi050 ceph-mon[30378]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:34.669+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) -- 0x7f826819eac0 con 0x7f824c0390c0 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:35.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:36 smithi050 ceph-mon[30378]: Updating smithi082:/etc/ceph/ceph.conf 2024-09-12T03:17:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:36 smithi050 ceph-mon[30378]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:36 smithi050 ceph-mon[30378]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:36 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:36 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:36 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:37 smithi050 ceph-mon[30378]: Deploying cephadm binary to smithi142 2024-09-12T03:17:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:37 smithi050 ceph-mon[30378]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:37 smithi050 ceph-mon[30378]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:38.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.658+0000 7f825fe00640 1 -- 172.21.15.50:0/2347142667 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f826819eac0 con 0x7f824c0390c0 2024-09-12T03:17:38.660 INFO:teuthology.orchestra.run.smithi050.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-09-12T03:17:38.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f824c0390c0 msgr2=0x7f824c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f824c0390c0 0x7f824c03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f82540098c0 tx=0x7f8254006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 msgr2=0x7f82681a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f82681a33d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8258009230 tx=0x7f8258003d40 comp rx=0 tx=0).stop 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 shutdown_connections 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f824c0390c0 0x7f824c03b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 --2- 172.21.15.50:0/2347142667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8268074590 0x7f82681a33d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 >> 172.21.15.50:0/2347142667 conn(0x7f82680fd230 msgr2=0x7f82680fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 shutdown_connections 2024-09-12T03:17:38.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:38.662+0000 7f826e600640 1 -- 172.21.15.50:0/2347142667 wait complete. 2024-09-12T03:17:39.344 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:39 smithi050 ceph-mon[30378]: Added host smithi142 2024-09-12T03:17:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:39 smithi050 ceph-mon[30378]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 -- 172.21.15.50:0/1984792920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 msgr2=0x7f5cd4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 --2- 172.21.15.50:0/1984792920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd4107a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009880 tx=0x7f5cc802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 -- 172.21.15.50:0/1984792920 shutdown_connections 2024-09-12T03:17:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 --2- 172.21.15.50:0/1984792920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd4107a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 -- 172.21.15.50:0/1984792920 >> 172.21.15.50:0/1984792920 conn(0x7f5cd40fd230 msgr2=0x7f5cd40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 -- 172.21.15.50:0/1984792920 shutdown_connections 2024-09-12T03:17:40.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.231+0000 7f5cdc400640 1 -- 172.21.15.50:0/1984792920 wait complete. 2024-09-12T03:17:40.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.232+0000 7f5cdc400640 1 Processor -- start 2024-09-12T03:17:40.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.232+0000 7f5cdc400640 1 -- start start 2024-09-12T03:17:40.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.232+0000 7f5cdc400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.232+0000 7f5cdc400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410bc70 con 0x7f5cd4107640 2024-09-12T03:17:40.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.233+0000 7f5cd9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.233+0000 7f5cd9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52092/0 (socket says 172.21.15.50:52092) 2024-09-12T03:17:40.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.233+0000 7f5cd9a00640 1 -- 172.21.15.50:0/3775486969 learned_addr learned my addr 172.21.15.50:0/3775486969 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.233+0000 7f5cd9a00640 1 -- 172.21.15.50:0/3775486969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd40763e0 con 0x7f5cd4107640 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.233+0000 7f5cd9a00640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd40744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009230 tx=0x7f5cc8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.234+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc803f070 con 0x7f5cd4107640 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.234+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc803bd40 con 0x7f5cd4107640 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.234+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8035030 con 0x7f5cd4107640 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.234+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd4074a30 con 0x7f5cd4107640 2024-09-12T03:17:40.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.234+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd4074ef0 con 0x7f5cd4107640 2024-09-12T03:17:40.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.235+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd4102cb0 con 0x7f5cd4107640 2024-09-12T03:17:40.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.235+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5cc8042070 con 0x7f5cd4107640 2024-09-12T03:17:40.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.236+0000 7f5cd2000640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cbc0390c0 0x7f5cbc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.236+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cc806c910 con 0x7f5cd4107640 2024-09-12T03:17:40.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.236+0000 7f5cd9000640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cbc0390c0 0x7f5cbc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.237+0000 7f5cd9000640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cbc0390c0 0x7f5cbc03b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00098c0 tx=0x7f5cc0006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.241+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cc8044e10 con 0x7f5cd4107640 2024-09-12T03:17:40.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.430+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cd4075a90 con 0x7f5cbc0390c0 2024-09-12T03:17:40.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.432+0000 7f5cd2000640 1 -- 172.21.15.50:0/3775486969 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f5cd4075a90 con 0x7f5cbc0390c0 2024-09-12T03:17:40.433 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:17:40.433 INFO:teuthology.orchestra.run.smithi050.stdout:[{"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cbc0390c0 msgr2=0x7f5cbc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cbc0390c0 0x7f5cbc03b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00098c0 tx=0x7f5cc0006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 msgr2=0x7f5cd40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd40744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009230 tx=0x7f5cc8003d80 comp rx=0 tx=0).stop 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 shutdown_connections 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cbc0390c0 0x7f5cbc03b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 --2- 172.21.15.50:0/3775486969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd4107640 0x7f5cd40744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 >> 172.21.15.50:0/3775486969 conn(0x7f5cd40fd230 msgr2=0x7f5cd40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 shutdown_connections 2024-09-12T03:17:40.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:40.434+0000 7f5cdc400640 1 -- 172.21.15.50:0/3775486969 wait complete. 2024-09-12T03:17:41.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:41 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:41.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:41 smithi050 ceph-mon[30378]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:41.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:41 smithi050 ceph-mon[30378]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:41.078 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T03:17:41.078 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T03:17:41.441 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:41.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 -- 172.21.15.50:0/955492861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 msgr2=0x7fc7c4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 --2- 172.21.15.50:0/955492861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4107a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009880 tx=0x7fc7b402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 -- 172.21.15.50:0/955492861 shutdown_connections 2024-09-12T03:17:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 --2- 172.21.15.50:0/955492861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4107a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 -- 172.21.15.50:0/955492861 >> 172.21.15.50:0/955492861 conn(0x7fc7c40fd290 msgr2=0x7fc7c40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 -- 172.21.15.50:0/955492861 shutdown_connections 2024-09-12T03:17:41.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.897+0000 7fc7c8e00640 1 -- 172.21.15.50:0/955492861 wait complete. 2024-09-12T03:17:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.898+0000 7fc7c8e00640 1 Processor -- start 2024-09-12T03:17:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.898+0000 7fc7c8e00640 1 -- start start 2024-09-12T03:17:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.899+0000 7fc7c8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.899+0000 7fc7c8e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c410bc90 con 0x7fc7c4107660 2024-09-12T03:17:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.899+0000 7fc7c1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.899+0000 7fc7c1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52114/0 (socket says 172.21.15.50:52114) 2024-09-12T03:17:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.899+0000 7fc7c1e00640 1 -- 172.21.15.50:0/3248035922 learned_addr learned my addr 172.21.15.50:0/3248035922 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:17:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.899+0000 7fc7c1e00640 1 -- 172.21.15.50:0/3248035922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c4197670 con 0x7fc7c4107660 2024-09-12T03:17:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.900+0000 7fc7c1e00640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4197130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009230 tx=0x7fc7b4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.900+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7b403f070 con 0x7fc7c4107660 2024-09-12T03:17:41.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.900+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b403bd40 con 0x7fc7c4107660 2024-09-12T03:17:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.900+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c4197900 con 0x7fc7c4107660 2024-09-12T03:17:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.900+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7b4035030 con 0x7fc7c4107660 2024-09-12T03:17:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.900+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c419ba00 con 0x7fc7c4107660 2024-09-12T03:17:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.901+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c4102cd0 con 0x7fc7c4107660 2024-09-12T03:17:41.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.902+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc7b4042070 con 0x7fc7c4107660 2024-09-12T03:17:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.902+0000 7fc7baa00640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc7a8039060 0x7fc7a803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.902+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc7b406c960 con 0x7fc7c4107660 2024-09-12T03:17:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.902+0000 7fc7c1400640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc7a8039060 0x7fc7a803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.903+0000 7fc7c1400640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc7a8039060 0x7fc7a803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00098c0 tx=0x7fc7b0006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:41.907+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7b40412c0 con 0x7fc7c4107660 2024-09-12T03:17:42.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.096+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc7c4198490 con 0x7fc7c4107660 2024-09-12T03:17:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.147+0000 7fc7baa00640 1 -- 172.21.15.50:0/3248035922 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc7b4044e10 con 0x7fc7c4107660 2024-09-12T03:17:42.148 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2024-09-12T03:17:42.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc7a8039060 msgr2=0x7fc7a803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc7a8039060 0x7fc7a803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00098c0 tx=0x7fc7b0006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 msgr2=0x7fc7c4197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4197130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009230 tx=0x7fc7b4003d80 comp rx=0 tx=0).stop 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 shutdown_connections 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc7a8039060 0x7fc7a803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 --2- 172.21.15.50:0/3248035922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc7c4107660 0x7fc7c4197130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.150+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 >> 172.21.15.50:0/3248035922 conn(0x7fc7c40fd290 msgr2=0x7fc7c40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.151+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 shutdown_connections 2024-09-12T03:17:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:17:42.151+0000 7fc7c8e00640 1 -- 172.21.15.50:0/3248035922 wait complete. 2024-09-12T03:17:42.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:42 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3248035922' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:17:42.823 INFO:tasks.cephadm:Adding mon.a on smithi050 2024-09-12T03:17:42.823 INFO:tasks.cephadm:Adding mon.b on smithi082 2024-09-12T03:17:42.823 INFO:tasks.cephadm:Adding mon.c on smithi142 2024-09-12T03:17:42.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c' 2024-09-12T03:17:43.067 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:17:43.113 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:17:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3248035922' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:17:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:17:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:43.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:43.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.572+0000 7f175684b640 1 -- 172.21.15.142:0/3904773861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 msgr2=0x7f17501044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.572+0000 7f175684b640 1 --2- 172.21.15.142:0/3904773861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f17501044c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1740009880 tx=0x7f174002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.572+0000 7f175684b640 1 -- 172.21.15.142:0/3904773861 shutdown_connections 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.572+0000 7f175684b640 1 --2- 172.21.15.142:0/3904773861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f17501044c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.572+0000 7f175684b640 1 -- 172.21.15.142:0/3904773861 >> 172.21.15.142:0/3904773861 conn(0x7f17500fbcd0 msgr2=0x7f17500fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.572+0000 7f175684b640 1 -- 172.21.15.142:0/3904773861 shutdown_connections 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.573+0000 7f175684b640 1 -- 172.21.15.142:0/3904773861 wait complete. 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.573+0000 7f175684b640 1 Processor -- start 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.573+0000 7f175684b640 1 -- start start 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.573+0000 7f175684b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f1750196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.573+0000 7f175684b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1750107670 con 0x7f17501020d0 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f1750196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f1750196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.142:37066/0 (socket says 172.21.15.142:37066) 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174ffff640 1 -- 172.21.15.142:0/539045912 learned_addr learned my addr 172.21.15.142:0/539045912 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174ffff640 1 -- 172.21.15.142:0/539045912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1750197250 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174ffff640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f1750196d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1740004f60 tx=0x7f17400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f174003f070 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1750197540 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1740003ad0 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1740035030 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.574+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175019a1c0 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.575+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1740042070 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.575+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1750107df0 con 0x7f17501020d0 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.575+0000 7f174d7fa640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1738039060 0x7f173803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.575+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f174006cf40 con 0x7f17501020d0 2024-09-12T03:17:43.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.576+0000 7f174f7fe640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1738039060 0x7f173803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.576+0000 7f174f7fe640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1738039060 0x7f173803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f17440098c0 tx=0x7f1744006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.581+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f174003be20 con 0x7f17501020d0 2024-09-12T03:17:43.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.763+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}) -- 0x7f175002d990 con 0x7f1738039060 2024-09-12T03:17:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.769+0000 7f174d7fa640 1 -- 172.21.15.142:0/539045912 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f175002d990 con 0x7f1738039060 2024-09-12T03:17:43.769 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mon update... 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.770+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1738039060 msgr2=0x7f173803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.770+0000 7f175684b640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1738039060 0x7f173803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f17440098c0 tx=0x7f1744006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.770+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 msgr2=0x7f1750196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.770+0000 7f175684b640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f1750196d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1740004f60 tx=0x7f17400048f0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.771+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 shutdown_connections 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.771+0000 7f175684b640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1738039060 0x7f173803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.771+0000 7f175684b640 1 --2- 172.21.15.142:0/539045912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17501020d0 0x7f1750196d10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.771+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 >> 172.21.15.142:0/539045912 conn(0x7f17500fbcd0 msgr2=0x7f17500fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.771+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 shutdown_connections 2024-09-12T03:17:43.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:43.771+0000 7f175684b640 1 -- 172.21.15.142:0/539045912 wait complete. 2024-09-12T03:17:44.357 DEBUG:teuthology.orchestra.run.smithi082:mon.b> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T03:17:44.360 DEBUG:teuthology.orchestra.run.smithi142:mon.c> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T03:17:44.363 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:17:44.363 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:17:44.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:44.633 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 -- 172.21.15.142:0/2220693639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 msgr2=0x7f1db0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 --2- 172.21.15.142:0/2220693639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0106920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009810 tx=0x7f1da002a540 comp rx=0 tx=0).stop 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 -- 172.21.15.142:0/2220693639 shutdown_connections 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 --2- 172.21.15.142:0/2220693639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0106920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 -- 172.21.15.142:0/2220693639 >> 172.21.15.142:0/2220693639 conn(0x7f1db0077310 msgr2=0x7f1db00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 -- 172.21.15.142:0/2220693639 shutdown_connections 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.024+0000 7f1db64cb640 1 -- 172.21.15.142:0/2220693639 wait complete. 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.025+0000 7f1db64cb640 1 Processor -- start 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.025+0000 7f1db64cb640 1 -- start start 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.025+0000 7f1db64cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.025+0000 7f1db64cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1db010ab70 con 0x7f1db0106540 2024-09-12T03:17:45.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.025+0000 7f1daffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1daffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.142:37094/0 (socket says 172.21.15.142:37094) 2024-09-12T03:17:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1daffff640 1 -- 172.21.15.142:0/705163627 learned_addr learned my addr 172.21.15.142:0/705163627 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:17:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1daffff640 1 -- 172.21.15.142:0/705163627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db01976c0 con 0x7f1db0106540 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1daffff640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1da00045e0 tx=0x7f1da0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da0030e70 con 0x7f1db0106540 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db0197950 con 0x7f1db0106540 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da00318e0 con 0x7f1db0106540 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da00049c0 con 0x7f1db0106540 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.026+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db0197db0 con 0x7f1db0106540 2024-09-12T03:17:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.027+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1da0039040 con 0x7f1db0106540 2024-09-12T03:17:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.027+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db0101bb0 con 0x7f1db0106540 2024-09-12T03:17:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.027+0000 7f1dadffb640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d9c0390c0 0x7f1d9c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.027+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1da006d410 con 0x7f1db0106540 2024-09-12T03:17:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.028+0000 7f1da7fff640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d9c0390c0 0x7f1d9c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.028+0000 7f1da7fff640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d9c0390c0 0x7f1d9c03b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d980098c0 tx=0x7f1d98006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.032+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1da0040030 con 0x7f1db0106540 2024-09-12T03:17:45.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:45 smithi050 ceph-mon[30378]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:45.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:45 smithi050 ceph-mon[30378]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c;count:3 2024-09-12T03:17:45.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:45 smithi050 ceph-mon[30378]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:17:45.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:45 smithi050 ceph-mon[30378]: Reconfiguring daemon mon.a on smithi050 2024-09-12T03:17:45.194 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:45 smithi050 ceph-mon[30378]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.265+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1db0198540 con 0x7f1db0106540 2024-09-12T03:17:45.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.266+0000 7f1dadffb640 1 -- 172.21.15.142:0/705163627 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1da0003730 con 0x7f1db0106540 2024-09-12T03:17:45.267 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:17:45.267 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:16:14.215514Z","created":"2024-09-12T03:16:14.215514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T03:17:45.267 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.268+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d9c0390c0 msgr2=0x7f1d9c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d9c0390c0 0x7f1d9c03b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d980098c0 tx=0x7f1d98006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 msgr2=0x7f1db0197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1da00045e0 tx=0x7f1da0004610 comp rx=0 tx=0).stop 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 shutdown_connections 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d9c0390c0 0x7f1d9c03b580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 --2- 172.21.15.142:0/705163627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1db0106540 0x7f1db0197180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 >> 172.21.15.142:0/705163627 conn(0x7f1db0077310 msgr2=0x7f1db00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 shutdown_connections 2024-09-12T03:17:45.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:45.269+0000 7f1db64cb640 1 -- 172.21.15.142:0/705163627 wait complete. 2024-09-12T03:17:46.566 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:46.566 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:46.566 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/705163627' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:46.567 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:46 smithi050 ceph-mon[30378]: Deploying daemon mon.c on smithi142 2024-09-12T03:17:46.884 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:17:46.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:17:47.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:47 smithi050 ceph-mon[30378]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:47.939 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 -- 172.21.15.142:0/2190870275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 msgr2=0x7fcfbc102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 --2- 172.21.15.142:0/2190870275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc102ac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009880 tx=0x7fcfac02a670 comp rx=0 tx=0).stop 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 -- 172.21.15.142:0/2190870275 shutdown_connections 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 --2- 172.21.15.142:0/2190870275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc102ac0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 -- 172.21.15.142:0/2190870275 >> 172.21.15.142:0/2190870275 conn(0x7fcfbc0fc0a0 msgr2=0x7fcfbc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 -- 172.21.15.142:0/2190870275 shutdown_connections 2024-09-12T03:17:48.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.524+0000 7fcfc13a0640 1 -- 172.21.15.142:0/2190870275 wait complete. 2024-09-12T03:17:48.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfc13a0640 1 Processor -- start 2024-09-12T03:17:48.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfc13a0640 1 -- start start 2024-09-12T03:17:48.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfc13a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfc13a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc10bbe0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.142:46492/0 (socket says 172.21.15.142:46492) 2024-09-12T03:17:48.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.525+0000 7fcfbbfff640 1 -- 172.21.15.142:0/967272900 learned_addr learned my addr 172.21.15.142:0/967272900 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:17:48.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfbbfff640 1 -- 172.21.15.142:0/967272900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc1976c0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfbbfff640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc197180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009230 tx=0x7fcfac003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfac03f070 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc197950 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfac03bc80 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfac035030 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.526+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc197db0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.527+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcfac042070 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.527+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc06acb0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.527+0000 7fcfb97fa640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcfa4039060 0x7fcfa403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.527+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcfac06caa0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.529 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.528+0000 7fcfbb7fe640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcfa4039060 0x7fcfa403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.529 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.528+0000 7fcfbb7fe640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcfa4039060 0x7fcfa403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00098c0 tx=0x7fcfb0006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.533+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfac031570 con 0x7fcfbc1026e0 2024-09-12T03:17:48.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.776+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcfbc1984e0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.777+0000 7fcfb97fa640 1 -- 172.21.15.142:0/967272900 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fcfac0372d0 con 0x7fcfbc1026e0 2024-09-12T03:17:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:17:48.777 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:16:14.215514Z","created":"2024-09-12T03:16:14.215514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T03:17:48.778 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-09-12T03:17:48.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcfa4039060 msgr2=0x7fcfa403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcfa4039060 0x7fcfa403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00098c0 tx=0x7fcfb0006db0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 msgr2=0x7fcfbc197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc197180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009230 tx=0x7fcfac003d40 comp rx=0 tx=0).stop 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 shutdown_connections 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcfa4039060 0x7fcfa403b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 --2- 172.21.15.142:0/967272900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfbc1026e0 0x7fcfbc197180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 >> 172.21.15.142:0/967272900 conn(0x7fcfbc0fc0a0 msgr2=0x7fcfbc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.779+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 shutdown_connections 2024-09-12T03:17:48.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:48.780+0000 7fcfc13a0640 1 -- 172.21.15.142:0/967272900 wait complete. 2024-09-12T03:17:49.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:48 smithi142 systemd[1]: Starting Ceph mon.c for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:17:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:48 smithi050 ceph-mon[30378]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:49.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 podman[35309]: 2024-09-12 03:17:49.613497189 +0000 UTC m=+0.016965619 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:17:49.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 podman[35309]: 2024-09-12 03:17:49.710962872 +0000 UTC m=+0.114431301 container create 53642584f51a6524a05e7e7011444ea6de40227e455a878e66c89627277a392e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:17:50.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:49 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/967272900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 podman[35309]: 2024-09-12 03:17:49.938194376 +0000 UTC m=+0.341662797 container init 53642584f51a6524a05e7e7011444ea6de40227e455a878e66c89627277a392e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 podman[35309]: 2024-09-12 03:17:49.941473097 +0000 UTC m=+0.344941517 container start 53642584f51a6524a05e7e7011444ea6de40227e455a878e66c89627277a392e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pidfile_write: ignore empty --pid-file 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: load: jerasure load: lrc 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Git sha 0 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:17:50.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: DB SUMMARY 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: DB Session ID: LRENOV4F7HKX8SWII10K 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.env: 0x557bc7d67940 2024-09-12T03:17:50.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:17:50.296 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.info_log: 0x557bca07d6e0 2024-09-12T03:17:50.296 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:17:50.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.statistics: (nil) 2024-09-12T03:17:50.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.use_fsync: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.db_log_dir: 2024-09-12T03:17:50.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.wal_dir: 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.write_buffer_manager: 0x557bca08cdc0 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:17:50.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.unordered_write: 0 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.row_cache: None 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.wal_filter: None 2024-09-12T03:17:50.300 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.wal_compression: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:17:50.301 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:17:50.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_open_files: -1 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Compression algorithms supported: 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kZSTD supported: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:17:50.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.merge_operator: 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_filter: None 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557bca07d320) 2024-09-12T03:17:50.304 INFO:journalctl@ceph.mon.c.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: index_type: 0 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: data_block_index_type: 0 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: index_shortening: 1 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: checksum: 4 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: no_block_cache: 0 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache: 0x557bca0711f0 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache_options: 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: capacity : 536870912 2024-09-12T03:17:50.305 INFO:journalctl@ceph.mon.c.smithi142.stdout: num_shard_bits : 4 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: strict_capacity_limit : 0 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_cache_compressed: (nil) 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: persistent_cache: (nil) 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_size: 4096 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_size_deviation: 10 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_restart_interval: 16 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: index_block_restart_interval: 1 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: metadata_block_size: 4096 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: partition_filters: 0 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: use_delta_encoding: 1 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: filter_policy: bloomfilter 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: whole_key_filtering: 1 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: verify_compression: 0 2024-09-12T03:17:50.306 INFO:journalctl@ceph.mon.c.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: format_version: 5 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: enable_index_compression: 1 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: block_align: 0 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: max_auto_readahead_size: 262144 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: prepopulate_block_cache: 0 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression: NoCompression 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:17:50.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.num_levels: 7 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:17:50.308 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:17:50.309 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:17:50.310 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:17:50.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.ttl: 2592000 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:17:50.312 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8d72c4c2-d034-451d-a5bd-7cfe3ebb535b 2024-09-12T03:17:50.313 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111069970245, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111069970658, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726111069, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8d72c4c2-d034-451d-a5bd-7cfe3ebb535b", "db_session_id": "LRENOV4F7HKX8SWII10K", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111069970760, "job": 1, "event": "recovery_finished"} 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557bca09ee00 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: DB pointer 0x557bca19a000 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** DB Stats ** 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:17:50.314 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** Compaction Stats [default] ** 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** Compaction Stats [default] ** 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:17:50.315 INFO:journalctl@ceph.mon.c.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:17:50.316 INFO:journalctl@ceph.mon.c.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: Block cache BinnedLRUCache@0x557bca0711f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: starting mon.c rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(???) e0 preinit fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: e1 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: btime 2024-09-12T03:16:17:849478+0000 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:17:50.317 INFO:journalctl@ceph.mon.c.smithi142.stdout: legacy client fscid: -1 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout: No filesystems configured 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi082:/etc/ceph/ceph.conf 2024-09-12T03:17:50.318 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Deploying cephadm binary to smithi142 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Added host smithi142 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.319 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3248035922' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3248035922' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:50.320 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:17:50.321 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c;count:3 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Reconfiguring daemon mon.a on smithi050 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/705163627' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:50.322 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: Deploying daemon mon.c on smithi142 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/967272900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:49 smithi142 ceph-mon[35329]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:50 smithi142 bash[35309]: 53642584f51a6524a05e7e7011444ea6de40227e455a878e66c89627277a392e 2024-09-12T03:17:50.323 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:50 smithi142 systemd[1]: Started Ceph mon.c for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:17:50.515 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:17:50.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:17:50.767 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: mon.a calling monitor election 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: mon.c calling monitor election 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: monmap epoch 2 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: last_changed 2024-09-12T03:17:49.998172+0000 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: min_mon_release 19 (squid) 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: election_strategy: 1 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: fsmap 2024-09-12T03:17:55.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: mgrmap e13: x(active, since 42s) 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: overall HEALTH_OK 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:55.417 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:55 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:55.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:17:55.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: mon.a calling monitor election 2024-09-12T03:17:55.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: mon.c calling monitor election 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: monmap epoch 2 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:55.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: last_changed 2024-09-12T03:17:49.998172+0000 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: min_mon_release 19 (squid) 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: election_strategy: 1 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: fsmap 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: mgrmap e13: x(active, since 42s) 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: overall HEALTH_OK 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:55.546 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:55.547 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:55.547 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:55 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:55.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 -- 172.21.15.142:0/2536382051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81066c0 msgr2=0x7ff2d0005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/2536382051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81066c0 0x7ff2d0005580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8012c60 tx=0x7ff2d802c620 comp rx=0 tx=0).stop 2024-09-12T03:17:55.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 -- 172.21.15.142:0/2536382051 shutdown_connections 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/2536382051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81066c0 0x7ff2d0005580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 -- 172.21.15.142:0/2536382051 >> 172.21.15.142:0/2536382051 conn(0x7ff2e80fb9a0 msgr2=0x7ff2e80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 -- 172.21.15.142:0/2536382051 shutdown_connections 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.609+0000 7ff2ecc62640 1 -- 172.21.15.142:0/2536382051 wait complete. 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.610+0000 7ff2ecc62640 1 Processor -- start 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.610+0000 7ff2ecc62640 1 -- start start 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.610+0000 7ff2ecc62640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.610+0000 7ff2ecc62640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81966c0 0x7ff2e819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.610+0000 7ff2ecc62640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8108720 con 0x7ff2e81966c0 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.610+0000 7ff2ecc62640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81085a0 con 0x7ff2e81066c0 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33586/0 (socket says 172.21.15.142:33586) 2024-09-12T03:17:55.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 -- 172.21.15.142:0/3315490687 learned_addr learned my addr 172.21.15.142:0/3315490687 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 -- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 msgr2=0x7ff2e8196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 -- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 msgr2=0x7ff2e8196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e5d9b640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81966c0 0x7ff2e819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e659c640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e5d9b640 1 -- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 msgr2=0x7ff2e8196180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e5d9b640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e5d9b640 1 -- 172.21.15.142:0/3315490687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e8196ce0 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.611+0000 7ff2e5d9b640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81966c0 0x7ff2e819aa50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00b3d0 tx=0x7ff2dc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.612+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff2dc011630 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.612+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2dc011cb0 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.612+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e819af90 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.612+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff2dc010df0 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.612+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e819b4d0 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.613+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff2dc018020 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.613+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e8101d10 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.613+0000 7ff2d77fe640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff2c8038e00 0x7ff2c803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.613+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff2dc04bc90 con 0x7ff2e81966c0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.614+0000 7ff2e659c640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff2c8038e00 0x7ff2c803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.614+0000 7ff2e659c640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff2c8038e00 0x7ff2c803b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8035040 tx=0x7ff2d8033660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.619+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2dc01f020 con 0x7ff2e81966c0 2024-09-12T03:17:55.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.856+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff2e818fe70 con 0x7ff2e81966c0 2024-09-12T03:17:55.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.856+0000 7ff2d77fe640 1 -- 172.21.15.142:0/3315490687 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ff2dc018310 con 0x7ff2e81966c0 2024-09-12T03:17:55.858 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:17:55.858 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:17:49.998172Z","created":"2024-09-12T03:16:14.215514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:17:55.858 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.858+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff2c8038e00 msgr2=0x7ff2c803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.858+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff2c8038e00 0x7ff2c803b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8035040 tx=0x7ff2d8033660 comp rx=0 tx=0).stop 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.858+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81966c0 msgr2=0x7ff2e819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.858+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81966c0 0x7ff2e819aa50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00b3d0 tx=0x7ff2dc00b890 comp rx=0 tx=0).stop 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.858+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 shutdown_connections 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.859+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff2c8038e00 0x7ff2c803b2c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.859+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2e81966c0 0x7ff2e819aa50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.859+0000 7ff2ecc62640 1 --2- 172.21.15.142:0/3315490687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2e81066c0 0x7ff2e8196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.859+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 >> 172.21.15.142:0/3315490687 conn(0x7ff2e80fb9a0 msgr2=0x7ff2e80fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.859+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 shutdown_connections 2024-09-12T03:17:55.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:55.859+0000 7ff2ecc62640 1 -- 172.21.15.142:0/3315490687 wait complete. 2024-09-12T03:17:56.210 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:56 smithi142 ceph-mon[35329]: Deploying daemon mon.b on smithi082 2024-09-12T03:17:56.211 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:56 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/3315490687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:56.211 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:56 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:56.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:56 smithi050 ceph-mon[30378]: Deploying daemon mon.b on smithi082 2024-09-12T03:17:56.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:56 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/3315490687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:56.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:56 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:57.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:57 smithi050 ceph-mon[30378]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:57.489 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:17:57.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:17:57.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:57 smithi142 ceph-mon[35329]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:57.739 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 -- 172.21.15.142:0/2430135846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df8102840 msgr2=0x7f9df8102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/2430135846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df8102840 0x7f9df8102c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009880 tx=0x7f9df402a670 comp rx=0 tx=0).stop 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 -- 172.21.15.142:0/2430135846 shutdown_connections 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/2430135846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df8102840 0x7f9df8102c20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 -- 172.21.15.142:0/2430135846 >> 172.21.15.142:0/2430135846 conn(0x7f9df80fc280 msgr2=0x7f9df80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 -- 172.21.15.142:0/2430135846 shutdown_connections 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.193+0000 7f9dfffc2640 1 -- 172.21.15.142:0/2430135846 wait complete. 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfffc2640 1 Processor -- start 2024-09-12T03:17:58.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfffc2640 1 -- start start 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfffc2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfffc2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df81979e0 0x7f9df819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfffc2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9df810bdb0 con 0x7f9df81979e0 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfffc2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9df810bc30 con 0x7f9df8102840 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33588/0 (socket says 172.21.15.142:33588) 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 -- 172.21.15.142:0/726872596 learned_addr learned my addr 172.21.15.142:0/726872596 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 -- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 msgr2=0x7f9df81974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 -- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 msgr2=0x7f9df81974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:17:58.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfdd5e640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.194+0000 7f9dfd55d640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df81979e0 0x7f9df819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9dfd55d640 1 -- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 msgr2=0x7f9df81974a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9dfd55d640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9dfd55d640 1 -- 172.21.15.142:0/726872596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df8198000 con 0x7f9df81979e0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9dfd55d640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df81979e0 0x7f9df819bd20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9de800b3d0 tx=0x7f9de800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9de8011630 con 0x7f9df81979e0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de8011cb0 con 0x7f9df81979e0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df819c260 con 0x7f9df81979e0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9de8010df0 con 0x7f9df81979e0 2024-09-12T03:17:58.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.195+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df819c7a0 con 0x7f9df81979e0 2024-09-12T03:17:58.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.196+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9de8018020 con 0x7f9df81979e0 2024-09-12T03:17:58.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.196+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df806a4f0 con 0x7f9df81979e0 2024-09-12T03:17:58.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.196+0000 7f9deeffd640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9de4038e50 0x7f9de403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.197+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9de804bc50 con 0x7f9df81979e0 2024-09-12T03:17:58.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.197+0000 7f9dfdd5e640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9de4038e50 0x7f9de403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.197+0000 7f9dfdd5e640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9de4038e50 0x7f9de403b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9df40096c0 tx=0x7f9df4005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:58.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.202+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9de801f020 con 0x7f9df81979e0 2024-09-12T03:17:58.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.431+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9df819cab0 con 0x7f9df81979e0 2024-09-12T03:17:58.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.432+0000 7f9deeffd640 1 -- 172.21.15.142:0/726872596 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f9de8017500 con 0x7f9df81979e0 2024-09-12T03:17:58.433 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:17:58.434 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:17:49.998172Z","created":"2024-09-12T03:16:14.215514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:17:58.434 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9de4038e50 msgr2=0x7f9de403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9de4038e50 0x7f9de403b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9df40096c0 tx=0x7f9df4005a40 comp rx=0 tx=0).stop 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df81979e0 msgr2=0x7f9df819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df81979e0 0x7f9df819bd20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9de800b3d0 tx=0x7f9de800b890 comp rx=0 tx=0).stop 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 shutdown_connections 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9de4038e50 0x7f9de403b310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9df81979e0 0x7f9df819bd20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 --2- 172.21.15.142:0/726872596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9df8102840 0x7f9df81974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 >> 172.21.15.142:0/726872596 conn(0x7f9df80fc280 msgr2=0x7f9df8105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:58.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 shutdown_connections 2024-09-12T03:17:58.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:17:58.435+0000 7f9dfffc2640 1 -- 172.21.15.142:0/726872596 wait complete. 2024-09-12T03:17:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:17:58 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/726872596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:58.807 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:17:58.807 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.info_log: 0x55d500cd56e0 2024-09-12T03:17:58.807 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:17:58.807 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.statistics: (nil) 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.use_fsync: 0 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:17:58.808 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:17:58.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:17:58 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/726872596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:17:59.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:17:59.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.db_log_dir: 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.wal_dir: 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:17:59.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.write_buffer_manager: 0x55d500ce4dc0 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.unordered_write: 0 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:17:59.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.row_cache: None 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.wal_filter: None 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.wal_compression: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:17:59.108 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:17:59.109 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_open_files: -1 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:17:59.110 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Compression algorithms supported: 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kZSTD supported: 0 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:17:59.111 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.merge_operator: 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_filter: None 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d500cd5320) 2024-09-12T03:17:59.112 INFO:journalctl@ceph.mon.b.smithi082.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: index_type: 0 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: data_block_index_type: 0 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: index_shortening: 1 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:17:59.113 INFO:journalctl@ceph.mon.b.smithi082.stdout: checksum: 4 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: no_block_cache: 0 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_cache: 0x55d500cc91f0 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_cache_options: 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: capacity : 536870912 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: num_shard_bits : 4 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: strict_capacity_limit : 0 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_cache_compressed: (nil) 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: persistent_cache: (nil) 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_size: 4096 2024-09-12T03:17:59.114 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_size_deviation: 10 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_restart_interval: 16 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: index_block_restart_interval: 1 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: metadata_block_size: 4096 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: partition_filters: 0 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: use_delta_encoding: 1 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: filter_policy: bloomfilter 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: whole_key_filtering: 1 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: verify_compression: 0 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: format_version: 5 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: enable_index_compression: 1 2024-09-12T03:17:59.115 INFO:journalctl@ceph.mon.b.smithi082.stdout: block_align: 0 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout: max_auto_readahead_size: 262144 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout: prepopulate_block_cache: 0 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression: NoCompression 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:17:59.116 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.num_levels: 7 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:17:59.117 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:17:59.118 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:17:59.119 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:17:59.120 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:17:59.120 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:17:59.120 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:17:59.120 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:17:59.122 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:17:59.122 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:17:59.122 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:17:59.122 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:17:59.122 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:17:59.123 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:17:59.124 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.ttl: 2592000 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:17:59.125 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f196d3c5-1399-4449-9f30-73eac1ebbc58 2024-09-12T03:17:59.126 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111078808079, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111078808485, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726111078, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f196d3c5-1399-4449-9f30-73eac1ebbc58", "db_session_id": "7TOBYLQK3G03TNZSLTW9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111078808579, "job": 1, "event": "recovery_finished"} 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d500cf6e00 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: DB pointer 0x55d500df2000 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** DB Stats ** 2024-09-12T03:17:59.127 INFO:journalctl@ceph.mon.b.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:17:59.128 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** Compaction Stats [default] ** 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-09-12T03:17:59.129 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** Compaction Stats [default] ** 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:17:59.130 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: Block cache BinnedLRUCache@0x55d500cc91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:17:59.131 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: using public_addr v2:172.21.15.82:0/0 -> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: starting mon.b rank -1 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(???) e0 preinit fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout: e1 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout: btime 2024-09-12T03:16:17:849478+0000 2024-09-12T03:17:59.132 INFO:journalctl@ceph.mon.b.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout: legacy client fscid: -1 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout: No filesystems configured 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:17:59.133 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi082:/etc/ceph/ceph.conf 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Deploying cephadm binary to smithi142 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Added host smithi142 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3248035922' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3248035922' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.136 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c;count:3 2024-09-12T03:17:59.137 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Reconfiguring daemon mon.a on smithi050 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/705163627' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:59.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Deploying daemon mon.c on smithi142 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/967272900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.a calling monitor election 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.c calling monitor election 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.139 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: monmap epoch 2 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: last_changed 2024-09-12T03:17:49.998172+0000 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: min_mon_release 19 (squid) 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: election_strategy: 1 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:17:59.140 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: fsmap 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mgrmap e13: x(active, since 42s) 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: overall HEALTH_OK 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:59.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: Deploying daemon mon.b on smithi082 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/3315490687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/726872596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 ceph-mon[35046]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 bash[35026]: 21880aeea3edbceac4bcb2da5b80ebf5f7ed71b3d67ec2826e57a46c4cd76dd7 2024-09-12T03:17:59.142 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:17:58 smithi082 systemd[1]: Started Ceph mon.b for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:18:00.028 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:18:00.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:18:00.280 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:18:03.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 -- 172.21.15.142:0/98501802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 msgr2=0x7f25e80059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 --2- 172.21.15.142:0/98501802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25e80059c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25fc06a240 tx=0x7f25f002c150 comp rx=0 tx=0).stop 2024-09-12T03:18:03.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 -- 172.21.15.142:0/98501802 shutdown_connections 2024-09-12T03:18:03.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 --2- 172.21.15.142:0/98501802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25e80059c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 -- 172.21.15.142:0/98501802 >> 172.21.15.142:0/98501802 conn(0x7f25fc0fc280 msgr2=0x7f25fc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:03.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 -- 172.21.15.142:0/98501802 shutdown_connections 2024-09-12T03:18:03.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.884+0000 7f26041d1640 1 -- 172.21.15.142:0/98501802 wait complete. 2024-09-12T03:18:03.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 Processor -- start 2024-09-12T03:18:03.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 -- start start 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25fc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 0x7f25fc19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 0x7f25fc19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc109750 con 0x7f25e8003530 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc1095d0 con 0x7f25fc1978e0 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.885+0000 7f26041d1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc1098d0 con 0x7f25fc19c1b0 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25fc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25fc1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.142:50112/0 (socket says 172.21.15.142:50112) 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260276e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 0x7f25fc19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 -- 172.21.15.142:0/555182209 learned_addr learned my addr 172.21.15.142:0/555182209 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260276e640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 msgr2=0x7f25fc19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T03:18:03.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260176c640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 0x7f25fc19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260276e640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 msgr2=0x7f25fc19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260276e640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 0x7f25fc19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260276e640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 0x7f25fc19c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260176c640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 msgr2=0x7f25fc19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260176c640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 msgr2=0x7f25fc19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260176c640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 0x7f25fc19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 msgr2=0x7f25fc19c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f260176c640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 0x7f25fc19bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 0x7f25fc19c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 msgr2=0x7f25fc19bc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 0x7f25fc19bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 -- 172.21.15.142:0/555182209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc1a3f20 con 0x7f25e8003530 2024-09-12T03:18:03.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.886+0000 7f2601f6d640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25fc1973a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25f00356a0 tx=0x7f25f00356d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:03.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.887+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0032040 con 0x7f25e8003530 2024-09-12T03:18:03.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.887+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f0043070 con 0x7f25e8003530 2024-09-12T03:18:03.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.887+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc1a40f0 con 0x7f25e8003530 2024-09-12T03:18:03.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.887+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f003e680 con 0x7f25e8003530 2024-09-12T03:18:03.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.887+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1a4650 con 0x7f25e8003530 2024-09-12T03:18:03.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.888+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f25f003e820 con 0x7f25e8003530 2024-09-12T03:18:03.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.888+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc10a880 con 0x7f25e8003530 2024-09-12T03:18:03.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.888+0000 7f25eeffd640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f25e00394f0 0x7f25e003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.888+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f25f006ded0 con 0x7f25e8003530 2024-09-12T03:18:03.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.888+0000 7f260176c640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f25e00394f0 0x7f25e003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.889+0000 7f260176c640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f25e00394f0 0x7f25e003b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25f8006fd0 tx=0x7f25f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:03.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:03.895+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25f003c2e0 con 0x7f25e8003530 2024-09-12T03:18:04.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.136+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f25fc1a4950 con 0x7f25e8003530 2024-09-12T03:18:04.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.136+0000 7f25eeffd640 1 -- 172.21.15.142:0/555182209 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f25f003a1d0 con 0x7f25e8003530 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: mon.a calling monitor election 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: mon.c calling monitor election 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:04.138 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: monmap epoch 3 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: last_changed 2024-09-12T03:17:58.835303+0000 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: min_mon_release 19 (squid) 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: election_strategy: 1 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: fsmap 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:18:04.139 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: mgrmap e13: x(active, since 51s) 2024-09-12T03:18:04.140 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: overall HEALTH_OK 2024-09-12T03:18:04.140 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.140 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.140 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.140 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.140 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:18:04.140 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":3,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:17:58.835303Z","created":"2024-09-12T03:16:14.215514Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:18:04.141 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 3 2024-09-12T03:18:04.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.139+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f25e00394f0 msgr2=0x7f25e003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.139+0000 7f26041d1640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f25e00394f0 0x7f25e003b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25f8006fd0 tx=0x7f25f8008040 comp rx=0 tx=0).stop 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.139+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 msgr2=0x7f25fc1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.139+0000 7f26041d1640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25fc1973a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25f00356a0 tx=0x7f25f00356d0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 shutdown_connections 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f25e00394f0 0x7f25e003b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25fc19c1b0 0x7f25fc19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25fc1978e0 0x7f25fc19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 --2- 172.21.15.142:0/555182209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25e8003530 0x7f25fc1973a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 >> 172.21.15.142:0/555182209 conn(0x7f25fc0fc280 msgr2=0x7f25fc0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 shutdown_connections 2024-09-12T03:18:04.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:04.140+0000 7f26041d1640 1 -- 172.21.15.142:0/555182209 wait complete. 2024-09-12T03:18:04.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:18:04.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: mon.a calling monitor election 2024-09-12T03:18:04.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: mon.c calling monitor election 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: monmap epoch 3 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: last_changed 2024-09-12T03:17:58.835303+0000 2024-09-12T03:18:04.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: min_mon_release 19 (squid) 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: election_strategy: 1 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: fsmap 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: mgrmap e13: x(active, since 51s) 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: overall HEALTH_OK 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:04.775 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T03:18:04.776 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T03:18:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:04 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/555182209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:18:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:04 smithi142 ceph-mon[35329]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:04 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/555182209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:18:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:04 smithi050 ceph-mon[30378]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.778+0000 7fdc3ec00640 1 -- 172.21.15.50:0/4104306476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 msgr2=0x7fdc38076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.778+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/4104306476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc38076b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009880 tx=0x7fdc2802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.778+0000 7fdc3ec00640 1 -- 172.21.15.50:0/4104306476 shutdown_connections 2024-09-12T03:18:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.779+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/4104306476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc38076b00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.779+0000 7fdc3ec00640 1 -- 172.21.15.50:0/4104306476 >> 172.21.15.50:0/4104306476 conn(0x7fdc380ff510 msgr2=0x7fdc38101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.779+0000 7fdc3ec00640 1 -- 172.21.15.50:0/4104306476 shutdown_connections 2024-09-12T03:18:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.779+0000 7fdc3ec00640 1 -- 172.21.15.50:0/4104306476 wait complete. 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 Processor -- start 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 -- start start 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc381a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 0x7fdc3819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3819de70 0x7fdc3819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdc38112380 con 0x7fdc38076720 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc38112200 con 0x7fdc381a3cb0 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.780+0000 7fdc3ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc38112500 con 0x7fdc3819de70 2024-09-12T03:18:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc381a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc381a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49236/0 (socket says 172.21.15.50:49236) 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 -- 172.21.15.50:0/2448773098 learned_addr learned my addr 172.21.15.50:0/2448773098 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc2fe00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 0x7fdc3819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc3cc00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3819de70 0x7fdc3819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc2fe00640 1 -- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 msgr2=0x7fdc3819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc2fe00640 1 -- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 msgr2=0x7fdc3819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc2fe00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 0x7fdc3819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc2fe00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 0x7fdc3819d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 -- 172.21.15.50:0/2448773098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3819de70 msgr2=0x7fdc3819e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3819de70 0x7fdc3819e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 -- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 msgr2=0x7fdc3819d930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 0x7fdc3819d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.781+0000 7fdc37e00640 1 -- 172.21.15.50:0/2448773098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc381aa640 con 0x7fdc38076720 2024-09-12T03:18:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.782+0000 7fdc37e00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc381a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc28006010 tx=0x7fdc28003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.782+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2803f070 con 0x7fdc38076720 2024-09-12T03:18:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.782+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2803b460 con 0x7fdc38076720 2024-09-12T03:18:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.782+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc381aa810 con 0x7fdc38076720 2024-09-12T03:18:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.782+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28035030 con 0x7fdc38076720 2024-09-12T03:18:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.782+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc381aad40 con 0x7fdc38076720 2024-09-12T03:18:05.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.783+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc38076b80 con 0x7fdc38076720 2024-09-12T03:18:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.784+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdc28042070 con 0x7fdc38076720 2024-09-12T03:18:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.784+0000 7fdc35600640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdc1c0394f0 0x7fdc1c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.784+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdc2806ccd0 con 0x7fdc38076720 2024-09-12T03:18:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.785+0000 7fdc2fe00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdc1c0394f0 0x7fdc1c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.785+0000 7fdc2fe00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdc1c0394f0 0x7fdc1c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc20006fd0 tx=0x7fdc20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:05.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.793+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc2804c350 con 0x7fdc38076720 2024-09-12T03:18:05.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.987+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdc381ab000 con 0x7fdc38076720 2024-09-12T03:18:05.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.987+0000 7fdc35600640 1 -- 172.21.15.50:0/2448773098 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fdc28036260 con 0x7fdc38076720 2024-09-12T03:18:05.989 INFO:teuthology.orchestra.run.smithi050.stdout:# minimal ceph.conf for 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:05.989 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2024-09-12T03:18:05.989 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:05.989 INFO:teuthology.orchestra.run.smithi050.stdout: mon_host = [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-09-12T03:18:05.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.989+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdc1c0394f0 msgr2=0x7fdc1c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.989+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdc1c0394f0 0x7fdc1c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc20006fd0 tx=0x7fdc20008040 comp rx=0 tx=0).stop 2024-09-12T03:18:05.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.989+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 msgr2=0x7fdc381a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.989+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc381a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc28006010 tx=0x7fdc28003c00 comp rx=0 tx=0).stop 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 shutdown_connections 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdc1c0394f0 0x7fdc1c03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3819de70 0x7fdc3819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381a3cb0 0x7fdc3819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 --2- 172.21.15.50:0/2448773098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc38076720 0x7fdc381a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 >> 172.21.15.50:0/2448773098 conn(0x7fdc380ff510 msgr2=0x7fdc380ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 shutdown_connections 2024-09-12T03:18:05.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:05.990+0000 7fdc3ec00640 1 -- 172.21.15.50:0/2448773098 wait complete. 2024-09-12T03:18:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:18:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: mon.a calling monitor election 2024-09-12T03:18:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: mon.c calling monitor election 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: monmap epoch 3 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:06.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: last_changed 2024-09-12T03:17:58.835303+0000 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: min_mon_release 19 (squid) 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: election_strategy: 1 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: fsmap 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: mgrmap e13: x(active, since 51s) 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: overall HEALTH_OK 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:06.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/555182209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:18:06.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:06.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: mon.b calling monitor election 2024-09-12T03:18:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: mon.b calling monitor election 2024-09-12T03:18:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: mon.c calling monitor election 2024-09-12T03:18:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: mon.a calling monitor election 2024-09-12T03:18:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: monmap epoch 3 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: last_changed 2024-09-12T03:17:58.835303+0000 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: min_mon_release 19 (squid) 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: election_strategy: 1 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: fsmap 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: mgrmap e13: x(active, since 53s) 2024-09-12T03:18:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: overall HEALTH_OK 2024-09-12T03:18:06.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2448773098' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:06.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: mon.b calling monitor election 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: mon.b calling monitor election 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: mon.c calling monitor election 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: mon.a calling monitor election 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: monmap epoch 3 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:06.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: last_changed 2024-09-12T03:17:58.835303+0000 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: min_mon_release 19 (squid) 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: election_strategy: 1 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: fsmap 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: mgrmap e13: x(active, since 53s) 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: overall HEALTH_OK 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2448773098' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:06.835 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T03:18:06.836 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:18:06.836 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:18:06.895 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:18:06.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:18:06.993 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:18:06.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:18:07.021 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:18:07.021 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:18:07.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: mon.b calling monitor election 2024-09-12T03:18:07.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: mon.b calling monitor election 2024-09-12T03:18:07.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: mon.c calling monitor election 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: mon.a calling monitor election 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: monmap epoch 3 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: last_changed 2024-09-12T03:17:58.835303+0000 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: created 2024-09-12T03:16:14.215514+0000 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: min_mon_release 19 (squid) 2024-09-12T03:18:07.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: election_strategy: 1 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: 2: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: fsmap 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: mgrmap e13: x(active, since 53s) 2024-09-12T03:18:07.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: overall HEALTH_OK 2024-09-12T03:18:07.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2448773098' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:07.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.088 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:18:07.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:18:07.121 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:18:07.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:18:07.191 INFO:tasks.cephadm:Adding mgr.x on smithi050 2024-09-12T03:18:07.191 INFO:tasks.cephadm:Adding mgr.y on smithi082 2024-09-12T03:18:07.192 INFO:tasks.cephadm:Adding mgr.z on smithi142 2024-09-12T03:18:07.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi050=x;smithi082=y;smithi142=z' 2024-09-12T03:18:07.474 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.821 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:07.822 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:07.822 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:07.822 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:07.822 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:07.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:07.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:07.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:07.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 -- 172.21.15.142:0/3993896831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081005f0 msgr2=0x7faf081009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 --2- 172.21.15.142:0/3993896831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081005f0 0x7faf081009f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faef8009880 tx=0x7faef802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 -- 172.21.15.142:0/3993896831 shutdown_connections 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 --2- 172.21.15.142:0/3993896831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081005f0 0x7faf081009f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 -- 172.21.15.142:0/3993896831 >> 172.21.15.142:0/3993896831 conn(0x7faf080fbda0 msgr2=0x7faf080fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 -- 172.21.15.142:0/3993896831 shutdown_connections 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.902+0000 7faf0da45640 1 -- 172.21.15.142:0/3993896831 wait complete. 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 Processor -- start 2024-09-12T03:18:07.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 -- start start 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf081005f0 0x7faf08196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 0x7faf0819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081979b0 0x7faf0819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf081066e0 con 0x7faf081979b0 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faf08106560 con 0x7faf081005f0 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf0da45640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faf08106860 con 0x7faf08197530 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.903+0000 7faf067fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 0x7faf0819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 0x7faf0819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43444/0 (socket says 172.21.15.142:43444) 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 -- 172.21.15.142:0/3772344064 learned_addr learned my addr 172.21.15.142:0/3772344064 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf077fe640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081979b0 0x7faf0819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:07.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf06ffd640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf081005f0 0x7faf08196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 -- 172.21.15.142:0/3772344064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf081005f0 msgr2=0x7faf08196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf081005f0 0x7faf08196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 -- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081979b0 msgr2=0x7faf0819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081979b0 0x7faf0819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 -- 172.21.15.142:0/3772344064 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0819f3f0 con 0x7faf08197530 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf06ffd640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf081005f0 0x7faf08196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf067fc640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 0x7faf0819c5a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faef000b520 tx=0x7faef000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0004230 con 0x7faf08197530 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef00043d0 con 0x7faf08197530 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0819f6e0 con 0x7faf08197530 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0003d60 con 0x7faf08197530 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.904+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0819fca0 con 0x7faf08197530 2024-09-12T03:18:07.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.905+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faef000e070 con 0x7faf08197530 2024-09-12T03:18:07.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.905+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0819ff60 con 0x7faf08197530 2024-09-12T03:18:07.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.905+0000 7faf0ca43640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faef40394f0 0x7faef403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:07.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.905+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faef004ba50 con 0x7faf08197530 2024-09-12T03:18:07.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.906+0000 7faf06ffd640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faef40394f0 0x7faef403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:07.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.906+0000 7faf06ffd640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faef40394f0 0x7faef403b9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faef8000c00 tx=0x7faef80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:07.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:07.910+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faef0014020 con 0x7faf08197530 2024-09-12T03:18:08.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.089+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi082=y;smithi142=z", "target": ["mon-mgr", ""]}) -- 0x7faf0806ab40 con 0x7faef40394f0 2024-09-12T03:18:08.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.100+0000 7faf0ca43640 1 -- 172.21.15.142:0/3772344064 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faf0806ab40 con 0x7faef40394f0 2024-09-12T03:18:08.101 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mgr update... 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faef40394f0 msgr2=0x7faef403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faef40394f0 0x7faef403b9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faef8000c00 tx=0x7faef80315b0 comp rx=0 tx=0).stop 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 msgr2=0x7faf0819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 0x7faf0819c5a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faef000b520 tx=0x7faef000b9e0 comp rx=0 tx=0).stop 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 shutdown_connections 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faef40394f0 0x7faef403b9b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:08.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf081979b0 0x7faf0819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:08.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf08197530 0x7faf0819c5a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:08.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 --2- 172.21.15.142:0/3772344064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf081005f0 0x7faf08196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:08.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 >> 172.21.15.142:0/3772344064 conn(0x7faf080fbda0 msgr2=0x7faf0806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:08.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 shutdown_connections 2024-09-12T03:18:08.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:18:08.103+0000 7faf0da45640 1 -- 172.21.15.142:0/3772344064 wait complete. 2024-09-12T03:18:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: Reconfiguring mon.a (monmap changed)... 2024-09-12T03:18:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: Reconfiguring daemon mon.a on smithi050 2024-09-12T03:18:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:08.671 DEBUG:teuthology.orchestra.run.smithi082:mgr.y> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T03:18:08.674 DEBUG:teuthology.orchestra.run.smithi142:mgr.z> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T03:18:08.676 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T03:18:08.676 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:18:08.677 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:18:08.712 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:18:08.712 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-09-12T03:18:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:18:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:08.789 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-12T03:18:08.789 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:08.789 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:08.789 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-12 03:17:25.344936830 +0000 2024-09-12T03:18:08.790 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-12 03:12:59.030924817 +0000 2024-09-12T03:18:08.790 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-12 03:12:59.030924817 +0000 2024-09-12T03:18:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-12 03:12:59.030924817 +0000 2024-09-12T03:18:08.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:18:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: Reconfiguring mon.a (monmap changed)... 2024-09-12T03:18:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: Reconfiguring daemon mon.a on smithi050 2024-09-12T03:18:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:08.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:08.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:08.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-12T03:18:08.890 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-12T03:18:08.890 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000209118 s, 2.4 MB/s 2024-09-12T03:18:08.892 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:18:08.932 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-12 03:17:25.388935668 +0000 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-12 03:12:59.497912361 +0000 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-12 03:12:59.497912361 +0000 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-12 03:12:59.497912361 +0000 2024-09-12T03:18:09.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:18:09.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: Reconfiguring mon.a (monmap changed)... 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: Reconfiguring daemon mon.a on smithi050 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:09.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:09.106 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-12T03:18:09.106 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-12T03:18:09.106 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000173181 s, 3.0 MB/s 2024-09-12T03:18:09.109 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:18:09.152 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-09-12T03:18:09.228 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:18:09.228 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-12 03:17:25.431934533 +0000 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-12 03:12:59.929900838 +0000 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-12 03:12:59.929900838 +0000 2024-09-12T03:18:09.229 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-12 03:12:59.929900838 +0000 2024-09-12T03:18:09.229 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:18:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-12T03:18:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-12T03:18:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000179555 s, 2.9 MB/s 2024-09-12T03:18:09.331 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:18:09.373 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-09-12T03:18:09.449 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-12 03:17:25.474933397 +0000 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-12 03:13:00.366889182 +0000 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-12 03:13:00.366889182 +0000 2024-09-12T03:18:09.450 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-12 03:13:00.366889182 +0000 2024-09-12T03:18:09.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:18:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-12T03:18:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-12T03:18:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000179132 s, 2.9 MB/s 2024-09-12T03:18:09.549 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:18:09.588 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-09-12T03:18:09.588 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:18:09.608 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:18:09.608 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-09-12T03:18:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:09 smithi050 ceph-mon[30378]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi082=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:09 smithi050 ceph-mon[30378]: Saving service mgr spec with placement smithi050=x;smithi082=y;smithi142=z;count:3 2024-09-12T03:18:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:09 smithi050 ceph-mon[30378]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:18:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:09 smithi050 ceph-mon[30378]: Reconfiguring daemon mgr.x on smithi050 2024-09-12T03:18:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:09 smithi050 ceph-mon[30378]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:09.667 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:18:09.667 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:09.667 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T03:18:09.667 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:09.668 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:09.668 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-09-12 03:17:58.280693892 +0000 2024-09-12T03:18:09.668 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-09-12 03:12:22.145582684 +0000 2024-09-12T03:18:09.668 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-09-12 03:12:22.145582684 +0000 2024-09-12T03:18:09.668 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-09-12 03:12:22.145582684 +0000 2024-09-12T03:18:09.669 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:18:09.735 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-09-12T03:18:09.735 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-09-12T03:18:09.735 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000295073 s, 1.7 MB/s 2024-09-12T03:18:09.736 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:18:09.796 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-09-12 03:17:58.280693892 +0000 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-09-12 03:12:22.476575756 +0000 2024-09-12T03:18:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-09-12 03:12:22.476575756 +0000 2024-09-12T03:18:09.855 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-09-12 03:12:22.476575756 +0000 2024-09-12T03:18:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:09 smithi082 ceph-mon[35046]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi082=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:09 smithi082 ceph-mon[35046]: Saving service mgr spec with placement smithi050=x;smithi082=y;smithi142=z;count:3 2024-09-12T03:18:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:09 smithi082 ceph-mon[35046]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:18:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:09 smithi082 ceph-mon[35046]: Reconfiguring daemon mgr.x on smithi050 2024-09-12T03:18:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:09 smithi082 ceph-mon[35046]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:09.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:18:09.922 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-09-12T03:18:09.922 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-09-12T03:18:09.923 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000215096 s, 2.4 MB/s 2024-09-12T03:18:09.924 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:18:09.981 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-09-12T03:18:10.038 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:18:10.038 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:10.038 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T03:18:10.039 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:10.039 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:10.039 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-09-12 03:17:58.281693871 +0000 2024-09-12T03:18:10.039 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-09-12 03:12:22.813568702 +0000 2024-09-12T03:18:10.039 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-09-12 03:12:22.813568702 +0000 2024-09-12T03:18:10.039 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-09-12 03:12:22.813568702 +0000 2024-09-12T03:18:10.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:18:10.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:09 smithi142 ceph-mon[35329]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi082=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:10.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:09 smithi142 ceph-mon[35329]: Saving service mgr spec with placement smithi050=x;smithi082=y;smithi142=z;count:3 2024-09-12T03:18:10.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:09 smithi142 ceph-mon[35329]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:18:10.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:09 smithi142 ceph-mon[35329]: Reconfiguring daemon mgr.x on smithi050 2024-09-12T03:18:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:09 smithi142 ceph-mon[35329]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:10.103 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-09-12T03:18:10.104 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-09-12T03:18:10.104 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000196937 s, 2.6 MB/s 2024-09-12T03:18:10.105 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:18:10.162 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-09-12T03:18:10.217 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:18:10.217 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:10.217 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T03:18:10.217 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:10.218 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:10.218 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-09-12 03:17:58.281693871 +0000 2024-09-12T03:18:10.218 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-09-12 03:12:23.134561984 +0000 2024-09-12T03:18:10.218 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-09-12 03:12:23.134561984 +0000 2024-09-12T03:18:10.218 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-09-12 03:12:23.134561984 +0000 2024-09-12T03:18:10.218 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:18:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-09-12T03:18:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-09-12T03:18:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000190772 s, 2.7 MB/s 2024-09-12T03:18:10.285 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:18:10.342 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T03:18:10.342 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:18:10.362 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:18:10.362 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 03:17:48.877011188 +0000 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 03:12:21.241239033 +0000 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 03:12:21.241239033 +0000 2024-09-12T03:18:10.421 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-12 03:12:21.241239033 +0000 2024-09-12T03:18:10.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:18:10.489 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T03:18:10.490 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T03:18:10.490 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000237605 s, 2.2 MB/s 2024-09-12T03:18:10.491 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:18:10.550 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-09-12T03:18:10.608 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 03:17:48.878011169 +0000 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 03:12:21.580232240 +0000 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 03:12:21.580232240 +0000 2024-09-12T03:18:10.609 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-12 03:12:21.580232240 +0000 2024-09-12T03:18:10.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:18:10.675 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T03:18:10.676 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T03:18:10.676 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000195594 s, 2.6 MB/s 2024-09-12T03:18:10.677 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:18:10.736 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-09-12T03:18:10.794 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:18:10.794 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:10.794 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T03:18:10.794 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:10.795 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:10.795 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 03:17:48.878011169 +0000 2024-09-12T03:18:10.795 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 03:12:21.910225627 +0000 2024-09-12T03:18:10.795 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 03:12:21.910225627 +0000 2024-09-12T03:18:10.795 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-12 03:12:21.910225627 +0000 2024-09-12T03:18:10.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:18:10.861 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T03:18:10.861 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T03:18:10.862 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00021968 s, 2.3 MB/s 2024-09-12T03:18:10.863 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:18:10.921 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-09-12T03:18:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:18:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:18:10.977 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T03:18:10.977 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:18:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:18:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 03:17:48.878011169 +0000 2024-09-12T03:18:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 03:12:22.238219055 +0000 2024-09-12T03:18:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 03:12:22.238219055 +0000 2024-09-12T03:18:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-12 03:12:22.238219055 +0000 2024-09-12T03:18:10.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:18:11.043 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T03:18:11.043 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T03:18:11.043 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000218511 s, 2.3 MB/s 2024-09-12T03:18:11.044 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:18:11.102 INFO:tasks.cephadm:Deploying osd.0 on smithi050 with /dev/vg_nvme/lv_4... 2024-09-12T03:18:11.102 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:18:11.409 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: Reconfiguring daemon mon.b on smithi082 2024-09-12T03:18:11.410 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:11 smithi082 ceph-mon[35046]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: Reconfiguring daemon mon.b on smithi082 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:11 smithi050 ceph-mon[30378]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:11.495 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:11.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:11.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: Reconfiguring daemon mon.b on smithi082 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:11 smithi142 ceph-mon[35329]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:12.483 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:12 smithi050 ceph-mon[30378]: Reconfiguring daemon mon.c on smithi142 2024-09-12T03:18:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:18:12.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:12.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:12.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:12.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:12 smithi142 ceph-mon[35329]: Reconfiguring daemon mon.c on smithi142 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:12 smithi082 ceph-mon[35046]: Reconfiguring daemon mon.c on smithi142 2024-09-12T03:18:13.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:13.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:13.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:13.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:13.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:18:13.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:13.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:14.093 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:18:14.125 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-09-12T03:18:14.492 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:14.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:14 smithi142 ceph-mon[35329]: Deploying daemon mgr.z on smithi142 2024-09-12T03:18:14.745 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:14 smithi050 ceph-mon[30378]: Deploying daemon mgr.z on smithi142 2024-09-12T03:18:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:14 smithi082 ceph-mon[35046]: Deploying daemon mgr.z on smithi142 2024-09-12T03:18:14.860 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:14 smithi142 podman[37120]: 2024-09-12 03:18:14.837037591 +0000 UTC m=+0.346096522 container start cdaa4d58097445a53e497292b12ff2604a987c28b4ac1b4cc46ee2683e90e1aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:18:14.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.948+0000 7f53ce600640 1 -- 172.21.15.50:0/3563595261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8077df0 msgr2=0x7f53c8078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:14.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.948+0000 7f53ce600640 1 --2- 172.21.15.50:0/3563595261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8077df0 0x7f53c8078250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009860 tx=0x7f53b002a340 comp rx=0 tx=0).stop 2024-09-12T03:18:14.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 -- 172.21.15.50:0/3563595261 shutdown_connections 2024-09-12T03:18:14.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 --2- 172.21.15.50:0/3563595261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8078790 0x7f53c8079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:14.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 --2- 172.21.15.50:0/3563595261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8077df0 0x7f53c8078250 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:14.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 --2- 172.21.15.50:0/3563595261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8076ba0 0x7f53c8076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:14.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 -- 172.21.15.50:0/3563595261 >> 172.21.15.50:0/3563595261 conn(0x7f53c80ff5b0 msgr2=0x7f53c81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:14.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 -- 172.21.15.50:0/3563595261 shutdown_connections 2024-09-12T03:18:14.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.949+0000 7f53ce600640 1 -- 172.21.15.50:0/3563595261 wait complete. 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 Processor -- start 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 -- start start 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8076ba0 0x7f53c819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 0x7f53c819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8078790 0x7f53c81a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8113e60 con 0x7f53c8077df0 2024-09-12T03:18:14.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8113ce0 con 0x7f53c8076ba0 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53ce600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8113fe0 con 0x7f53c8078790 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 0x7f53c819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 0x7f53c819ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47196/0 (socket says 172.21.15.50:47196) 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.950+0000 7f53bfe00640 1 -- 172.21.15.50:0/2252581472 learned_addr learned my addr 172.21.15.50:0/2252581472 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53c7e00640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8078790 0x7f53c81a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53c7400640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8076ba0 0x7f53c819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53bfe00640 1 -- 172.21.15.50:0/2252581472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8078790 msgr2=0x7f53c81a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53bfe00640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8078790 0x7f53c81a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53bfe00640 1 -- 172.21.15.50:0/2252581472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8076ba0 msgr2=0x7f53c819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53bfe00640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8076ba0 0x7f53c819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:14.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53bfe00640 1 -- 172.21.15.50:0/2252581472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81a6120 con 0x7f53c8077df0 2024-09-12T03:18:14.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53bfe00640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 0x7f53c819ea30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53b0035040 tx=0x7f53b0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:14.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.951+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0037040 con 0x7f53c8077df0 2024-09-12T03:18:14.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.952+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b0032770 con 0x7f53c8077df0 2024-09-12T03:18:14.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.952+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c81a63b0 con 0x7f53c8077df0 2024-09-12T03:18:14.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.952+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00046c0 con 0x7f53c8077df0 2024-09-12T03:18:14.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.952+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81a6890 con 0x7f53c8077df0 2024-09-12T03:18:14.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.953+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f53b0004050 con 0x7f53c8077df0 2024-09-12T03:18:14.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.953+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c8110780 con 0x7f53c8077df0 2024-09-12T03:18:14.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.953+0000 7f53c4c00640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f53ac0392c0 0x7f53ac03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:14.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.953+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53b006c300 con 0x7f53c8077df0 2024-09-12T03:18:14.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.953+0000 7f53c7400640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f53ac0392c0 0x7f53ac03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:14.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.954+0000 7f53c7400640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f53ac0392c0 0x7f53ac03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005fd0 tx=0x7f53b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:14.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:14.958+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53b0004340 con 0x7f53c8077df0 2024-09-12T03:18:15.165 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:14 smithi142 ceph-mgr[37139]: -- 172.21.15.142:0/2981145815 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9553390e0 con 0x55f955317400 2024-09-12T03:18:15.166 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:14 smithi142 bash[37120]: cdaa4d58097445a53e497292b12ff2604a987c28b4ac1b4cc46ee2683e90e1aa 2024-09-12T03:18:15.166 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:14 smithi142 systemd[1]: Started Ceph mgr.z for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:18:15.166 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:14.957+0000 7ff279932040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:18:15.166 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.018+0000 7ff279932040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:18:15.166 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.165+0000 7ff279932040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:18:15.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:15.166+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f53c8062630 con 0x7f53ac0392c0 2024-09-12T03:18:15.478 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.231+0000 7ff279932040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:18:15.478 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.377+0000 7ff279932040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:18:15.711 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:15.711 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:18:15.712 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:18:15.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:15.757 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:15.757 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:18:15.757 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:18:15.757 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:15.793 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.516+0000 7ff279932040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:18:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:18:15.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:16.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.825+0000 7ff279932040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:18:16.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:15.885+0000 7ff279932040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:18:16.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:16.066+0000 7ff279932040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:18:16.517 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:16 smithi082 ceph-mon[35046]: Deploying daemon mgr.y on smithi082 2024-09-12T03:18:16.517 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:16 smithi082 ceph-mon[35046]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:16.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:16 smithi050 ceph-mon[30378]: Deploying daemon mgr.y on smithi082 2024-09-12T03:18:16.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:16 smithi050 ceph-mon[30378]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:16 smithi142 ceph-mon[35329]: Deploying daemon mgr.y on smithi082 2024-09-12T03:18:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:16 smithi142 ceph-mon[35329]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:16.854 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:16 smithi082 systemd[1]: Starting Ceph mgr.y for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:18:17.155 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:16 smithi082 podman[36397]: 2024-09-12 03:18:16.795718344 +0000 UTC m=+0.015501685 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:17.156 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:16 smithi082 podman[36397]: 2024-09-12 03:18:16.925180064 +0000 UTC m=+0.144963442 container create 349fd9d888ab90fc482fedd42a292b46fe2e9b2c384279ad5f49702c75689a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:18:17.156 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 podman[36397]: 2024-09-12 03:18:17.156180622 +0000 UTC m=+0.375963964 container init 349fd9d888ab90fc482fedd42a292b46fe2e9b2c384279ad5f49702c75689a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:18:17.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:16.937+0000 7ff279932040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:18:17.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:17.003+0000 7ff279932040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:18:17.483 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 podman[36397]: 2024-09-12 03:18:17.15901824 +0000 UTC m=+0.378801581 container start 349fd9d888ab90fc482fedd42a292b46fe2e9b2c384279ad5f49702c75689a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T03:18:17.484 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mgr[36417]: -- 172.21.15.82:0/1209059186 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55698dd0b0e0 con 0x55698dce9000 2024-09-12T03:18:17.484 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 bash[36397]: 349fd9d888ab90fc482fedd42a292b46fe2e9b2c384279ad5f49702c75689a45 2024-09-12T03:18:17.484 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 systemd[1]: Started Ceph mgr.y for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:18:17.484 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:17.285+0000 7f1e6465e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:18:17.484 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:17.347+0000 7f1e6465e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:18:17.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:17 smithi050 ceph-mon[30378]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:17.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:17.736 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-mon[35329]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:17.736 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.736 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.736 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.736 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.736 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:17.854 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:17.501+0000 7f1e6465e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:18:17.854 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:17.571+0000 7f1e6465e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:18:17.854 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:17.726+0000 7f1e6465e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:18:17.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mon[35046]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:17.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:17.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:18.015 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:17.736+0000 7ff279932040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:18:18.015 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:17.864+0000 7ff279932040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:18:18.015 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.015+0000 7ff279932040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:18:18.170 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:17.882+0000 7f1e6465e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:18:18.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.076+0000 7ff279932040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:18:18.293 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.234+0000 7ff279932040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:18:18.456 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:18.207+0000 7f1e6465e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:18:18.456 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:18.268+0000 7f1e6465e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:18:18.602 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.304+0000 7ff279932040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:18:18.602 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.363+0000 7ff279932040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:18:18.602 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.533+0000 7ff279932040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:18:18.602 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.601+0000 7ff279932040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:18:18.708 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:18.455+0000 7f1e6465e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/290852865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "650c6e13-7a10-4d87-abd0-ed5b66b22521"}]: dispatch 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/290852865' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "650c6e13-7a10-4d87-abd0-ed5b66b22521"}]': finished 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:18.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:18.763 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.763 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:18 smithi050 ceph-mon[30378]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:18.876 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.668+0000 7ff279932040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:18:18.876 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.754+0000 7ff279932040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:18:18.876 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.813+0000 7ff279932040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:18:18.876 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:18.876+0000 7ff279932040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/290852865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "650c6e13-7a10-4d87-abd0-ed5b66b22521"}]: dispatch 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/290852865' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "650c6e13-7a10-4d87-abd0-ed5b66b22521"}]': finished 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:18.877 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:18 smithi142 ceph-mon[35329]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/290852865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "650c6e13-7a10-4d87-abd0-ed5b66b22521"}]: dispatch 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/290852865' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "650c6e13-7a10-4d87-abd0-ed5b66b22521"}]': finished 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:19.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:18 smithi082 ceph-mon[35046]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:19.710 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:19.360+0000 7f1e6465e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:18:19.710 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:19.426+0000 7f1e6465e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:18:20.043 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:18:19.660+0000 7ff279932040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:18:20.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:20.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:20.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:20.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:20.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2268508729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:18:20.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: Standby manager daemon z started 2024-09-12T03:18:20.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:18:20.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:18:20.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:18:20.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:19 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:18:20.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:20.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2268508729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: Standby manager daemon z started 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:18:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:19 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:18:20.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:20.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:20.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:20.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:20.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2268508729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:18:20.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: Standby manager daemon z started 2024-09-12T03:18:20.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:18:20.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:18:20.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:18:20.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:19 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/1164449985' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:18:20.401 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.144+0000 7f1e6465e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:18:20.401 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.259+0000 7f1e6465e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:18:20.401 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.401+0000 7f1e6465e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:18:20.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:20.575+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f53b002fd40 con 0x7f53c8077df0 2024-09-12T03:18:20.678 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.462+0000 7f1e6465e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:18:20.678 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.611+0000 7f1e6465e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:18:20.678 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.678+0000 7f1e6465e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:18:20.959 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.737+0000 7f1e6465e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:18:20.960 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.895+0000 7f1e6465e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:18:20.960 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:20.959+0000 7f1e6465e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:18:21.225 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:21.025+0000 7f1e6465e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:18:21.225 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:21.107+0000 7f1e6465e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:18:21.225 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:21.165+0000 7f1e6465e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:18:21.225 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:21.225+0000 7f1e6465e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:18:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-mon[35046]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-12T03:18:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:18:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:21 smithi082 ceph-mon[35046]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:21.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:21 smithi050 ceph-mon[30378]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-12T03:18:21.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:18:21.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:21 smithi050 ceph-mon[30378]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:22.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:21 smithi142 ceph-mon[35329]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-12T03:18:22.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:18:22.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:21 smithi142 ceph-mon[35329]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:22.354 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:18:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:18:22.015+0000 7f1e6465e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:18:22.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:22.588+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f53b0031b80 con 0x7f53c8077df0 2024-09-12T03:18:22.842 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:22 smithi050 ceph-mon[30378]: Standby manager daemon y started 2024-09-12T03:18:22.842 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:22 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:18:22.843 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:22 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:18:22.843 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:22 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:18:22.843 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:22 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:18:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:22 smithi082 ceph-mon[35046]: Standby manager daemon y started 2024-09-12T03:18:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:22 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:18:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:22 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:18:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:22 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:18:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:22 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:18:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:22 smithi142 ceph-mon[35329]: Standby manager daemon y started 2024-09-12T03:18:23.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:22 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:18:23.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:22 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:18:23.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:22 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:18:23.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:22 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/2747026648' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:18:23.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:23 smithi082 ceph-mon[35046]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-12T03:18:23.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:18:23.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:23 smithi082 ceph-mon[35046]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:23.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:23 smithi050 ceph-mon[30378]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-12T03:18:23.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:18:23.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:23 smithi050 ceph-mon[30378]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:24.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:23 smithi142 ceph-mon[35329]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-12T03:18:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:18:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:23 smithi142 ceph-mon[35329]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:24.978 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:24 smithi050 ceph-mon[30378]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:24 smithi142 ceph-mon[35329]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:24 smithi082 ceph-mon[35046]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:26.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:26 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:18:26.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:26 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:26.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:26 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:18:26.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:26 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:26 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:18:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:26 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:27.574 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:27 smithi050 ceph-mon[30378]: Deploying daemon osd.0 on smithi050 2024-09-12T03:18:27.574 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:27 smithi050 ceph-mon[30378]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:27.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:27 smithi142 ceph-mon[35329]: Deploying daemon osd.0 on smithi050 2024-09-12T03:18:27.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:27 smithi142 ceph-mon[35329]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:27.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:27 smithi082 ceph-mon[35046]: Deploying daemon osd.0 on smithi050 2024-09-12T03:18:27.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:27 smithi082 ceph-mon[35046]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:28 smithi142 ceph-mon[35329]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:29.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:28 smithi050 ceph-mon[30378]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:28 smithi082 ceph-mon[35046]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:30 smithi142 ceph-mon[35329]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:30 smithi082 ceph-mon[35046]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:30 smithi050 ceph-mon[30378]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:32 smithi050 ceph-mon[30378]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:32 smithi142 ceph-mon[35329]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:32 smithi082 ceph-mon[35046]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:34.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:34.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:34.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:34.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:34.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:34.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:34.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:34.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:34.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:35 smithi050 ceph-mon[30378]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:35.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:35 smithi142 ceph-mon[35329]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:35.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:35 smithi082 ceph-mon[35046]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:37.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:37 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:37 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:37 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:37.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:37 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:37.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:37 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:37 smithi142 ceph-mon[35329]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:37 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:37 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:37 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:37 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:37 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:37 smithi082 ceph-mon[35046]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:37 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:37 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:37 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:37 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:37.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:37 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:37.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:37 smithi050 ceph-mon[30378]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:38.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.339+0000 7f53c4c00640 1 -- 172.21.15.50:0/2252581472 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53c8062630 con 0x7f53ac0392c0 2024-09-12T03:18:38.340 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 0 on host 'smithi050' 2024-09-12T03:18:38.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f53ac0392c0 msgr2=0x7f53ac03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f53ac0392c0 0x7f53ac03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005fd0 tx=0x7f53b8009260 comp rx=0 tx=0).stop 2024-09-12T03:18:38.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 msgr2=0x7f53c819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 0x7f53c819ea30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53b0035040 tx=0x7f53b0033d90 comp rx=0 tx=0).stop 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 shutdown_connections 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f53ac0392c0 0x7f53ac03b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8078790 0x7f53c81a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53c8077df0 0x7f53c819ea30 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53b0035040 tx=0x7f53b0033d90 comp rx=0 tx=0).stop 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 --2- 172.21.15.50:0/2252581472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53c8076ba0 0x7f53c819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.342+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 >> 172.21.15.50:0/2252581472 conn(0x7f53c80ff5b0 msgr2=0x7f53c8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.343+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 shutdown_connections 2024-09-12T03:18:38.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:18:38.343+0000 7f53ce600640 1 -- 172.21.15.50:0/2252581472 wait complete. 2024-09-12T03:18:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:39.302 DEBUG:teuthology.orchestra.run.smithi050:osd.0> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T03:18:39.307 INFO:tasks.cephadm:Deploying osd.1 on smithi082 with /dev/vg_nvme/lv_4... 2024-09-12T03:18:39.307 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:18:39.566 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:18:39.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:39 smithi050 ceph-mon[30378]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:39.914 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:18:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0[44361]: 2024-09-12T03:18:39.721+0000 7f6363e2f640 -1 osd.0 0 log_to_monitors true 2024-09-12T03:18:39.930 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:39 smithi082 ceph-mon[35046]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:40.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:39 smithi142 ceph-mon[35329]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:40.697 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:40 smithi050 ceph-mon[30378]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:18:40.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:40 smithi082 ceph-mon[35046]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:18:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:40 smithi142 ceph-mon[35329]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:18:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:41 smithi082 ceph-mon[35046]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:18:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:41 smithi082 ceph-mon[35046]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:18:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:41 smithi082 ceph-mon[35046]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:18:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:41 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:41 smithi082 ceph-mon[35046]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:41.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-mon[30378]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:18:41.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-mon[30378]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:18:41.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-mon[30378]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:18:41.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:41.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-mon[30378]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:41.915 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0[44361]: 2024-09-12T03:18:41.673+0000 7f635e600640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T03:18:41.916 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:18:41 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0[44361]: 2024-09-12T03:18:41.676+0000 7f6359600640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:18:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:18:41.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-09-12T03:18:42.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:41 smithi142 ceph-mon[35329]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:18:42.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:41 smithi142 ceph-mon[35329]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:18:42.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:41 smithi142 ceph-mon[35329]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:18:42.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:41 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:42.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:41 smithi142 ceph-mon[35329]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:42.244 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 -- 172.21.15.82:0/3194010480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 msgr2=0x7f40a8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 --2- 172.21.15.82:0/3194010480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a8108f20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ca0 tx=0x7f40a002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 -- 172.21.15.82:0/3194010480 shutdown_connections 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 --2- 172.21.15.82:0/3194010480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a8108f20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 --2- 172.21.15.82:0/3194010480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 0x7f40a8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 --2- 172.21.15.82:0/3194010480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 0x7f40a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.710+0000 7f40acb15640 1 -- 172.21.15.82:0/3194010480 >> 172.21.15.82:0/3194010480 conn(0x7f40a80fc280 msgr2=0x7f40a80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.711+0000 7f40acb15640 1 -- 172.21.15.82:0/3194010480 shutdown_connections 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.711+0000 7f40acb15640 1 -- 172.21.15.82:0/3194010480 wait complete. 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.711+0000 7f40acb15640 1 Processor -- start 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.711+0000 7f40acb15640 1 -- start start 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 0x7f40a8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 0x7f40a8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40a810c750 con 0x7f40a8100ad0 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40a810c5d0 con 0x7f40a8102690 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40a810c8d0 con 0x7f40a8101cd0 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57174/0 (socket says 172.21.15.82:57174) 2024-09-12T03:18:42.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 -- 172.21.15.82:0/1211283043 learned_addr learned my addr 172.21.15.82:0/1211283043 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f409dd9b640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 0x7f40a8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a659c640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 0x7f40a8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 -- 172.21.15.82:0/1211283043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 msgr2=0x7f40a8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 0x7f40a8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 -- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 msgr2=0x7f40a8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 0x7f40a8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 -- 172.21.15.82:0/1211283043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a819faf0 con 0x7f40a8102690 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f409dd9b640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 0x7f40a8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a659c640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 0x7f40a8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40a6d9d640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a819f480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40a0005e30 tx=0x7f40a0031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0003c30 con 0x7f40a8102690 2024-09-12T03:18:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a0003dd0 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a819fd80 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.712+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a002fde0 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.713+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a81a0260 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.713+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a81a0490 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.715+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f40a0004050 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.715+0000 7f409ffff640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f40940a2b20 0x7f40940a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.715+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f40a00eea70 con 0x7f40a8102690 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.716+0000 7f40a659c640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f40940a2b20 0x7f40940a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.716+0000 7f40a659c640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f40940a2b20 0x7f40940a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40980058c0 tx=0x7f4098005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.719+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40a00bd440 con 0x7f40a8102690 2024-09-12T03:18:42.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:18:42.897+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f40a81a0660 con 0x7f40940a2b20 2024-09-12T03:18:42.898 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:18:42.898 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:18:42.898 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:42.898 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:42.898 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:42.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:42.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:42.899 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:18:42.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:18:42.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:42.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:43.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:43.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:43.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:43.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:18:43.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi050 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi050 to 17094M 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] boot 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:18:43.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:44.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi050 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi050 to 17094M 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] boot 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:18:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:44.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi050 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi050 to 17094M 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: osd.0 [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] boot 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:18:44.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:44.831 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:44 smithi082 ceph-mon[35046]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:44.831 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:44 smithi082 ceph-mon[35046]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:18:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:44 smithi142 ceph-mon[35329]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:44 smithi142 ceph-mon[35329]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:18:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:44 smithi050 ceph-mon[30378]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:44 smithi050 ceph-mon[30378]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:18:46.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:45 smithi142 ceph-mon[35329]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:45 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/2592485056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]: dispatch 2024-09-12T03:18:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:45 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]: dispatch 2024-09-12T03:18:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:45 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]': finished 2024-09-12T03:18:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:45 smithi142 ceph-mon[35329]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:18:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:45 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:18:46.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:45 smithi050 ceph-mon[30378]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:46.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:45 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/2592485056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]: dispatch 2024-09-12T03:18:46.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:45 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]: dispatch 2024-09-12T03:18:46.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:45 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]': finished 2024-09-12T03:18:46.067 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:45 smithi050 ceph-mon[30378]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:18:46.067 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:45 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:18:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:45 smithi082 ceph-mon[35046]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:46.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:45 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/2592485056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]: dispatch 2024-09-12T03:18:46.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:45 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]: dispatch 2024-09-12T03:18:46.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:45 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "989d2e6e-def7-4a97-a83f-da0168e02a90"}]': finished 2024-09-12T03:18:46.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:45 smithi082 ceph-mon[35046]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:18:46.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:45 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:18:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:46 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/3644607917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:18:47.078 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:46 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/3644607917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:18:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:46 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/3644607917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:18:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:47 smithi142 ceph-mon[35329]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:47 smithi082 ceph-mon[35046]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:48.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:47 smithi050 ceph-mon[30378]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:48 smithi142 ceph-mon[35329]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:48 smithi082 ceph-mon[35046]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:48 smithi050 ceph-mon[30378]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:50.873 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:50 smithi082 ceph-mon[35046]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:51.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:50 smithi142 ceph-mon[35329]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:50 smithi050 ceph-mon[30378]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:53.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:52 smithi142 ceph-mon[35329]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:53.112 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:52 smithi082 ceph-mon[35046]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:53.173 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:52 smithi050 ceph-mon[30378]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:53.991 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:53 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:18:53.991 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:53 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:53.991 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:53 smithi082 ceph-mon[35046]: Deploying daemon osd.1 on smithi082 2024-09-12T03:18:54.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:53 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:18:54.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:53 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:54.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:53 smithi142 ceph-mon[35329]: Deploying daemon osd.1 on smithi082 2024-09-12T03:18:54.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:53 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:18:54.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:53 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:54.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:53 smithi050 ceph-mon[30378]: Deploying daemon osd.1 on smithi082 2024-09-12T03:18:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:54 smithi142 ceph-mon[35329]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:55.051 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:54 smithi082 ceph-mon[35046]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:55.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:54 smithi050 ceph-mon[30378]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:56.862 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:56 smithi082 ceph-mon[35046]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:57.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:56 smithi142 ceph-mon[35329]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:57.138 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:56 smithi050 ceph-mon[30378]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:58 smithi082 ceph-mon[35046]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:58 smithi142 ceph-mon[35329]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:58 smithi050 ceph-mon[30378]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:18:59.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:18:59.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:18:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:18:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:00.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:00.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:00.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:18:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:01.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:00 smithi082 ceph-mon[35046]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:00 smithi050 ceph-mon[30378]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:01.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:00 smithi142 ceph-mon[35329]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:03.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.173+0000 7f409ffff640 1 -- 172.21.15.82:0/1211283043 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f40a81a0660 con 0x7f40940a2b20 2024-09-12T03:19:03.173 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 1 on host 'smithi082' 2024-09-12T03:19:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f40940a2b20 msgr2=0x7f40940a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f40940a2b20 0x7f40940a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40980058c0 tx=0x7f4098005850 comp rx=0 tx=0).stop 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 msgr2=0x7f40a819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a819f480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40a0005e30 tx=0x7f40a0031ee0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 shutdown_connections 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f40940a2b20 0x7f40940a4fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a8102690 0x7f40a819f480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a8101cd0 0x7f40a8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 --2- 172.21.15.82:0/1211283043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40a8100ad0 0x7f40a8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 >> 172.21.15.82:0/1211283043 conn(0x7f40a80fc280 msgr2=0x7f40a8069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 shutdown_connections 2024-09-12T03:19:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:19:03.175+0000 7f40acb15640 1 -- 172.21.15.82:0/1211283043 wait complete. 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.829 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.830 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.857 DEBUG:teuthology.orchestra.run.smithi082:osd.1> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T03:19:03.860 INFO:tasks.cephadm:Deploying osd.2 on smithi142 with /dev/vg_nvme/lv_4... 2024-09-12T03:19:03.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:19:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:04.111 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:19:05.016 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:04 smithi082 ceph-mon[35046]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:04 smithi142 ceph-mon[35329]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:04 smithi050 ceph-mon[30378]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:06.325 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:06 smithi082 ceph-mon[35046]: from='osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:19:06.326 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:06 smithi082 ceph-mon[35046]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:19:06.326 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1[39517]: 2024-09-12T03:19:06.122+0000 7f8acc775640 -1 osd.1 0 log_to_monitors true 2024-09-12T03:19:06.470 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:06 smithi142 ceph-mon[35329]: from='osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:19:06.470 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:06 smithi142 ceph-mon[35329]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:19:06.472 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:19:06.497 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-09-12T03:19:06.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:06 smithi050 ceph-mon[30378]: from='osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:19:06.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:06 smithi050 ceph-mon[30378]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:19:06.761 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:19:07.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 -- 172.21.15.142:0/1781317222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210101cd0 msgr2=0x7f0210102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 --2- 172.21.15.142:0/1781317222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210101cd0 0x7f0210102150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0204009900 tx=0x7f020402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 -- 172.21.15.142:0/1781317222 shutdown_connections 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 --2- 172.21.15.142:0/1781317222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0210102690 0x7f0210108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 --2- 172.21.15.142:0/1781317222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210101cd0 0x7f0210102150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 --2- 172.21.15.142:0/1781317222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210100ad0 0x7f0210100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 -- 172.21.15.142:0/1781317222 >> 172.21.15.142:0/1781317222 conn(0x7f02100fc280 msgr2=0x7f02100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 -- 172.21.15.142:0/1781317222 shutdown_connections 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.217+0000 7f0215197640 1 -- 172.21.15.142:0/1781317222 wait complete. 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 Processor -- start 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 -- start start 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0210100ad0 0x7f0210196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210101cd0 0x7f0210197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 0x7f021019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f021010c750 con 0x7f0210101cd0 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f021010c5d0 con 0x7f0210100ad0 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f0215197640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f021010c8d0 con 0x7f0210102690 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 0x7f021019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 0x7f021019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:51158/0 (socket says 172.21.15.142:51158) 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 -- 172.21.15.142:0/320432453 learned_addr learned my addr 172.21.15.142:0/320432453 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020ed9d640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0210100ad0 0x7f0210196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 -- 172.21.15.142:0/320432453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0210100ad0 msgr2=0x7f0210196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0210100ad0 0x7f0210196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 -- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210101cd0 msgr2=0x7f0210197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210101cd0 0x7f0210197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.218+0000 7f020f59e640 1 -- 172.21.15.142:0/320432453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0210069ff0 con 0x7f0210102690 2024-09-12T03:19:07.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.219+0000 7f020f59e640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 0x7f021019e300 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f020800eb10 tx=0x7f020800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.219+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208004b20 con 0x7f0210102690 2024-09-12T03:19:07.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.219+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0208004cc0 con 0x7f0210102690 2024-09-12T03:19:07.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.219+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021006a2e0 con 0x7f0210102690 2024-09-12T03:19:07.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.219+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208010760 con 0x7f0210102690 2024-09-12T03:19:07.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.219+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021006a7d0 con 0x7f0210102690 2024-09-12T03:19:07.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.220+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0210107e00 con 0x7f0210102690 2024-09-12T03:19:07.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.221+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0208005000 con 0x7f0210102690 2024-09-12T03:19:07.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.222+0000 7f01f7fff640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f80a2b20 0x7f01f80a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.222+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f02080cde60 con 0x7f0210102690 2024-09-12T03:19:07.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.222+0000 7f020ed9d640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f80a2b20 0x7f01f80a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.223+0000 7f020ed9d640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f80a2b20 0x7f01f80a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02000058c0 tx=0x7f0200005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.230+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f020809c730 con 0x7f0210102690 2024-09-12T03:19:07.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:07.411+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f021006aba0 con 0x7f01f80a2b20 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi082 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi082 to 17088M 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.412 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:07.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:07.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.413 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:07 smithi142 ceph-mon[35329]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:07.604 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1[39517]: 2024-09-12T03:19:07.191+0000 7f8ac86f8640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T03:19:07.604 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1[39517]: 2024-09-12T03:19:07.195+0000 7f8ac4520640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:19:07.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:19:07.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:19:07.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi082 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi082 to 17088M 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:07 smithi082 ceph-mon[35046]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:07.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi082 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi082 to 17088M 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:07.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:07.666 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:07 smithi050 ceph-mon[30378]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:08.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:19:08.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:19:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:19:08.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:19:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:19:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:19:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:19:08.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:09.232 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:09 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:19:09.233 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:09 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:19:09.233 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:09 smithi142 ceph-mon[35329]: osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] boot 2024-09-12T03:19:09.233 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:09 smithi142 ceph-mon[35329]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:19:09.233 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:09.233 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:09 smithi142 ceph-mon[35329]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:09 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:19:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:09 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:19:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:09 smithi082 ceph-mon[35046]: osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] boot 2024-09-12T03:19:09.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:09 smithi082 ceph-mon[35046]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:19:09.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:09.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:09 smithi082 ceph-mon[35046]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:09 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:19:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:09 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:19:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:09 smithi050 ceph-mon[30378]: osd.1 [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] boot 2024-09-12T03:19:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:09 smithi050 ceph-mon[30378]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:19:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:19:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:09 smithi050 ceph-mon[30378]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:19:10.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:10 smithi142 ceph-mon[35329]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:19:10.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:10 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/358278269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]: dispatch 2024-09-12T03:19:10.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:10 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]: dispatch 2024-09-12T03:19:10.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:10 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]': finished 2024-09-12T03:19:10.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:10 smithi142 ceph-mon[35329]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:19:10.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:10 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:10 smithi082 ceph-mon[35046]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:19:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:10 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/358278269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]: dispatch 2024-09-12T03:19:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:10 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]: dispatch 2024-09-12T03:19:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:10 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]': finished 2024-09-12T03:19:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:10 smithi082 ceph-mon[35046]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:19:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:10 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:10.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:10 smithi050 ceph-mon[30378]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:19:10.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:10 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/358278269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]: dispatch 2024-09-12T03:19:10.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:10 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]: dispatch 2024-09-12T03:19:10.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:10 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5565f10-86c0-4440-9256-ae2f1729cfb6"}]': finished 2024-09-12T03:19:10.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:10 smithi050 ceph-mon[30378]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:19:10.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:10 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:11.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:11 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/150959131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:19:11.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:11 smithi142 ceph-mon[35329]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:11.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:11 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/150959131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:19:11.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:11 smithi082 ceph-mon[35046]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:11 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/150959131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:19:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:11 smithi050 ceph-mon[30378]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:13.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:12 smithi142 ceph-mon[35329]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:12 smithi082 ceph-mon[35046]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:12 smithi050 ceph-mon[30378]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:14.938 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:14 smithi142 ceph-mon[35329]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:14 smithi082 ceph-mon[35046]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:14 smithi050 ceph-mon[30378]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:16.953 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:16 smithi142 ceph-mon[35329]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:16 smithi082 ceph-mon[35046]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:16 smithi050 ceph-mon[30378]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:18.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:19:18.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:18.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:19:18.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:18.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:19:18.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:19.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:19 smithi142 ceph-mon[35329]: Deploying daemon osd.2 on smithi142 2024-09-12T03:19:19.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:19 smithi142 ceph-mon[35329]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:19 smithi082 ceph-mon[35046]: Deploying daemon osd.2 on smithi142 2024-09-12T03:19:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:19 smithi082 ceph-mon[35046]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:19.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:19 smithi050 ceph-mon[30378]: Deploying daemon osd.2 on smithi142 2024-09-12T03:19:19.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:19 smithi050 ceph-mon[30378]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:21.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:20 smithi142 ceph-mon[35329]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:20 smithi082 ceph-mon[35046]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:20 smithi050 ceph-mon[30378]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:23.029 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:22 smithi142 ceph-mon[35329]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:22 smithi082 ceph-mon[35046]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:22 smithi050 ceph-mon[30378]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:24.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:24.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:24.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:24.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:24.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:24.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:25.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:25 smithi142 ceph-mon[35329]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:25.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:25 smithi082 ceph-mon[35046]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:25.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:25 smithi050 ceph-mon[30378]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:27.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:26 smithi142 ceph-mon[35329]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:26 smithi082 ceph-mon[35046]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:26 smithi050 ceph-mon[30378]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:28.067 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:27 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.067 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:27 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.067 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:27 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:28.067 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:27 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:28.067 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:27 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:27 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:27 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:27 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:27 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:27 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:28.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:28.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.446+0000 7f01f7fff640 1 -- 172.21.15.142:0/320432453 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f021006aba0 con 0x7f01f80a2b20 2024-09-12T03:19:28.449 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 2 on host 'smithi142' 2024-09-12T03:19:28.449 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f80a2b20 msgr2=0x7f01f80a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.449 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f80a2b20 0x7f01f80a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02000058c0 tx=0x7f0200005850 comp rx=0 tx=0).stop 2024-09-12T03:19:28.449 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 msgr2=0x7f021019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 0x7f021019e300 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f020800eb10 tx=0x7f020800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 shutdown_connections 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f80a2b20 0x7f01f80a4fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210102690 0x7f021019e300 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210101cd0 0x7f0210197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 --2- 172.21.15.142:0/320432453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0210100ad0 0x7f0210196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.448+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 >> 172.21.15.142:0/320432453 conn(0x7f02100fc280 msgr2=0x7f0210104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.449+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 shutdown_connections 2024-09-12T03:19:28.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:19:28.449+0000 7f0215197640 1 -- 172.21.15.142:0/320432453 wait complete. 2024-09-12T03:19:29.025 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:29.025 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:29.025 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:29.025 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:28 smithi142 ceph-mon[35329]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:29.053 DEBUG:teuthology.orchestra.run.smithi142:osd.2> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T03:19:29.056 INFO:tasks.cephadm:Deploying osd.3 on smithi050 with /dev/vg_nvme/lv_3... 2024-09-12T03:19:29.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:19:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:28 smithi082 ceph-mon[35046]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:29.110 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:29.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:29.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:29.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:28 smithi050 ceph-mon[30378]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:29.434 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:30.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:30 smithi142 ceph-mon[35329]: from='osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:19:30.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:30 smithi142 ceph-mon[35329]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:19:30.794 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2[40268]: 2024-09-12T03:19:30.522+0000 7f631b906640 -1 osd.2 0 log_to_monitors true 2024-09-12T03:19:30.813 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:30 smithi050 ceph-mon[30378]: from='osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:19:30.813 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:30 smithi050 ceph-mon[30378]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:19:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:30 smithi082 ceph-mon[35046]: from='osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:19:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:30 smithi082 ceph-mon[35046]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:19:31.793 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2[40268]: 2024-09-12T03:19:31.591+0000 7f6317889640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T03:19:31.794 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2[40268]: 2024-09-12T03:19:31.594+0000 7f63136b1640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:19:31.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-mon[35329]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:19:31.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-mon[35329]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:19:31.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:31.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-mon[35329]: from='osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:19:31.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-mon[35329]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:19:31.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:31 smithi142 ceph-mon[35329]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:31.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:31 smithi082 ceph-mon[35046]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:19:31.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:31 smithi082 ceph-mon[35046]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:19:31.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:31 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:31.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:31 smithi082 ceph-mon[35046]: from='osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:19:31.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:31 smithi082 ceph-mon[35046]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:19:31.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:31 smithi082 ceph-mon[35046]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:31.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:31 smithi050 ceph-mon[30378]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:19:31.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:31 smithi050 ceph-mon[30378]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:19:31.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:31 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:31.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:31 smithi050 ceph-mon[30378]: from='osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:19:31.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:31 smithi050 ceph-mon[30378]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:19:31.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:31 smithi050 ceph-mon[30378]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:32.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.769 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:19:32.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-09-12T03:19:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:19:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:19:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:32.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:32.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:32.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:32.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:33.169 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:33.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 -- 172.21.15.50:0/1943824298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4103e00 msgr2=0x7f61d4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 --2- 172.21.15.50:0/1943824298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4103e00 0x7f61d4104200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009860 tx=0x7f61c402a340 comp rx=0 tx=0).stop 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 -- 172.21.15.50:0/1943824298 shutdown_connections 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 --2- 172.21.15.50:0/1943824298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4105920 0x7f61d410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 --2- 172.21.15.50:0/1943824298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d41053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 --2- 172.21.15.50:0/1943824298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4103e00 0x7f61d4104200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 -- 172.21.15.50:0/1943824298 >> 172.21.15.50:0/1943824298 conn(0x7f61d40ff5b0 msgr2=0x7f61d41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 -- 172.21.15.50:0/1943824298 shutdown_connections 2024-09-12T03:19:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.831+0000 7f61d9200640 1 -- 172.21.15.50:0/1943824298 wait complete. 2024-09-12T03:19:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 Processor -- start 2024-09-12T03:19:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 -- start start 2024-09-12T03:19:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4103e00 0x7f61d419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4105920 0x7f61d41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410fab0 con 0x7f61d4104f60 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410f930 con 0x7f61d4105920 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.832+0000 7f61d9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410fc30 con 0x7f61d4103e00 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34252/0 (socket says 172.21.15.50:34252) 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 -- 172.21.15.50:0/3155971258 learned_addr learned my addr 172.21.15.50:0/3155971258 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61d2800640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4105920 0x7f61d41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61d1e00640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4103e00 0x7f61d419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 -- 172.21.15.50:0/3155971258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4103e00 msgr2=0x7f61d419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4103e00 0x7f61d419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 -- 172.21.15.50:0/3155971258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4105920 msgr2=0x7f61d41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4105920 0x7f61d41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61c9400640 1 -- 172.21.15.50:0/3155971258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d41a1d00 con 0x7f61d4104f60 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.833+0000 7f61d1e00640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4103e00 0x7f61d419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.834+0000 7f61c9400640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d419a610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00e9f0 tx=0x7f61bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.834+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00ccc0 con 0x7f61d4104f60 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.834+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61bc0035f0 con 0x7f61d4104f60 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.834+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00ce60 con 0x7f61d4104f60 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.834+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d41a1ff0 con 0x7f61d4104f60 2024-09-12T03:19:33.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.834+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d41a2450 con 0x7f61d4104f60 2024-09-12T03:19:33.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.835+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d410b190 con 0x7f61d4104f60 2024-09-12T03:19:33.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.837+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f61bc0037a0 con 0x7f61d4104f60 2024-09-12T03:19:33.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.839+0000 7f61cb400640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f61b80a2ba0 0x7f61b80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.839+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f61bc0cde00 con 0x7f61d4104f60 2024-09-12T03:19:33.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.839+0000 7f61d1e00640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f61b80a2ba0 0x7f61b80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi142 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] boot 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:33.840 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:33.841 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:33 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:33.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.840+0000 7f61d1e00640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f61b80a2ba0 0x7f61b80a5060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009830 tx=0x7f61c4030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:33.842+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61bc09c210 con 0x7f61d4104f60 2024-09-12T03:19:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:19:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:19:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi142 2024-09-12T03:19:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-12T03:19:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] boot 2024-09-12T03:19:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:19:33.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:33.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:33.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:33 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:34.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:34.035+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f61d4062630 con 0x7f61b80a2ba0 2024-09-12T03:19:34.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:19:34.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi142 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: osd.2 [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] boot 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:19:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:33 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:34 smithi050 ceph-mon[30378]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:19:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:19:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:19:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:34 smithi142 ceph-mon[35329]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:19:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:19:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:19:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:34 smithi082 ceph-mon[35046]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:19:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:19:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:19:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:35.517 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48681]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:19:35.517 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48681]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:35.517 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48681]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:35.517 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48681]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:35.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:35.622+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f61bc098180 con 0x7f61d4104f60 2024-09-12T03:19:35.822 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:35 smithi050 ceph-mon[30378]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:35.823 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:35.823 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:35 smithi050 ceph-mon[30378]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T03:19:35.823 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:35 smithi050 ceph-mon[30378]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:35 smithi142 ceph-mon[35329]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:35 smithi142 ceph-mon[35329]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T03:19:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:35 smithi142 ceph-mon[35329]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:35 smithi082 ceph-mon[35046]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:35 smithi082 ceph-mon[35046]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T03:19:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:35 smithi082 ceph-mon[35046]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:36.115 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48686]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:19:36.115 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48686]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:36.115 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48686]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:36.115 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:19:35 smithi050 sudo[48686]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:36.374 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:19:36.374 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:36.374 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43200]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:36.374 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43200]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:36.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:36 smithi050 ceph-mon[30378]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T03:19:36.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:36 smithi050 ceph-mon[30378]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:19:36.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:36 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:36.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:36 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:36 smithi082 ceph-mon[35046]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T03:19:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:36 smithi082 ceph-mon[35046]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:19:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:36 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:36 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:36.854 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:19:36.855 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:36.855 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43203]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:36.855 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:19:36 smithi082 sudo[43203]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:36.898 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:36 smithi142 ceph-mon[35329]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T03:19:36.898 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:36 smithi142 ceph-mon[35329]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:19:36.898 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:36 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:36.899 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:36 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:36.899 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44347]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:19:36.899 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:36.899 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44347]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:36.899 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44347]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:37.294 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44350]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:19:37.294 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:37.294 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44350]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:37.294 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:19:36 smithi142 sudo[44350]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:37 smithi050 sudo[48759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:19:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:37 smithi050 sudo[48759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:37 smithi050 sudo[48759]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:37.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:37 smithi050 sudo[48759]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:38.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 ceph-mon[35329]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:38.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:38.304 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 ceph-mon[35329]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:38.305 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44356]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:19:38.305 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44356]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:38.305 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44356]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:38.305 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44356]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:38.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 ceph-mon[35046]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:38.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:38.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 ceph-mon[35046]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 sudo[48762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 sudo[48762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 sudo[48762]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 sudo[48762]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 ceph-mon[30378]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:38.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:38 smithi050 ceph-mon[30378]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:38.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44359]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:19:38.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44359]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:38.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44359]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:38.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:38 smithi142 sudo[44359]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:39.042 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 sudo[43208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:19:39.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 sudo[43208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:39.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 sudo[43208]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:39.043 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:38 smithi082 sudo[43208]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/256697722' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]': finished 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:39 smithi050 ceph-mon[30378]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:39.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 sudo[43211]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 sudo[43211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 sudo[43211]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 sudo[43211]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/256697722' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]: dispatch 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]: dispatch 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.617 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]': finished 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.618 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:39 smithi082 ceph-mon[35046]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:39.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/256697722' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]: dispatch 2024-09-12T03:19:39.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:39.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a331fa56-9a0a-4823-bd51-3d23adb41a13"}]': finished 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:19:39.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:39.621 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:39.621 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:19:39.621 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:19:39.621 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:39 smithi142 ceph-mon[35329]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:40.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:40 smithi082 ceph-mon[35046]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:40.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:40 smithi082 ceph-mon[35046]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:40.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:40 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1041476304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:19:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:40 smithi050 ceph-mon[30378]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:40.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:40 smithi050 ceph-mon[30378]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:40.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:40 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1041476304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:19:40.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:40 smithi142 ceph-mon[35329]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:19:40.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:40 smithi142 ceph-mon[35329]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:19:40.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:40 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1041476304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:19:41.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:41 smithi082 ceph-mon[35046]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:41 smithi050 ceph-mon[30378]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:41.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:41 smithi142 ceph-mon[35329]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:43.039 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:42 smithi050 ceph-mon[30378]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:42 smithi142 ceph-mon[35329]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:42 smithi082 ceph-mon[35046]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:44.952 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:44 smithi050 ceph-mon[30378]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:44 smithi142 ceph-mon[35329]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:44 smithi082 ceph-mon[35046]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:46 smithi142 ceph-mon[35329]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:47.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:46 smithi082 ceph-mon[35046]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:47.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:46 smithi050 ceph-mon[30378]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:19:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:47 smithi050 ceph-mon[30378]: Deploying daemon osd.3 on smithi050 2024-09-12T03:19:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:19:48.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:48.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:47 smithi142 ceph-mon[35329]: Deploying daemon osd.3 on smithi050 2024-09-12T03:19:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:19:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:48.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:47 smithi082 ceph-mon[35046]: Deploying daemon osd.3 on smithi050 2024-09-12T03:19:48.859 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:48 smithi050 ceph-mon[30378]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:48 smithi142 ceph-mon[35329]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:48 smithi082 ceph-mon[35046]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:50.878 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:50 smithi050 ceph-mon[30378]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:51.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:50 smithi142 ceph-mon[35329]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:50 smithi082 ceph-mon[35046]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:53.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:52 smithi142 ceph-mon[35329]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:53.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:52 smithi050 ceph-mon[30378]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:53.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:52 smithi082 ceph-mon[35046]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:54 smithi050 ceph-mon[30378]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:54 smithi142 ceph-mon[35329]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:55.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:55.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:55.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:54 smithi082 ceph-mon[35046]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:56.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:56 smithi050 ceph-mon[30378]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:57.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:56 smithi142 ceph-mon[35329]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:57.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:56 smithi082 ceph-mon[35046]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:58.754 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:58.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:58.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:58.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:58.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:58.908 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:58.909 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:58.909 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:58.909 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:58.909 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:59.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:59.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:59.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:59.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:59.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.090+0000 7f61cb400640 1 -- 172.21.15.50:0/3155971258 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61d4062630 con 0x7f61b80a2ba0 2024-09-12T03:19:59.091 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 3 on host 'smithi050' 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f61b80a2ba0 msgr2=0x7f61b80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f61b80a2ba0 0x7f61b80a5060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009830 tx=0x7f61c4030550 comp rx=0 tx=0).stop 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 msgr2=0x7f61d419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d419a610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00e9f0 tx=0x7f61bc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 shutdown_connections 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f61b80a2ba0 0x7f61b80a5060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61d4105920 0x7f61d41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d4104f60 0x7f61d419a610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 --2- 172.21.15.50:0/3155971258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d4103e00 0x7f61d419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 >> 172.21.15.50:0/3155971258 conn(0x7f61d40ff5b0 msgr2=0x7f61d4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 shutdown_connections 2024-09-12T03:19:59.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:19:59.093+0000 7f61d9200640 1 -- 172.21.15.50:0/3155971258 wait complete. 2024-09-12T03:19:59.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:59 smithi050 ceph-mon[30378]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:19:59.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:59.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:59.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:19:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:19:59.819 DEBUG:teuthology.orchestra.run.smithi050:osd.3> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T03:19:59.824 INFO:tasks.cephadm:Deploying osd.4 on smithi082 with /dev/vg_nvme/lv_3... 2024-09-12T03:19:59.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:20:00.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:59 smithi142 ceph-mon[35329]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:00.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:00.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:00.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:19:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:20:00.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:59 smithi082 ceph-mon[35046]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:00.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:00.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:00.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:19:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:00 smithi082 ceph-mon[35046]: overall HEALTH_OK 2024-09-12T03:20:00.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:00 smithi050 ceph-mon[30378]: overall HEALTH_OK 2024-09-12T03:20:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:00 smithi142 ceph-mon[35329]: overall HEALTH_OK 2024-09-12T03:20:01.374 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:20:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3[50827]: 2024-09-12T03:20:00.961+0000 7f786f68c640 -1 osd.3 0 log_to_monitors true 2024-09-12T03:20:01.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:01 smithi082 ceph-mon[35046]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:01.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:01 smithi082 ceph-mon[35046]: from='osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:20:01.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:01 smithi082 ceph-mon[35046]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:20:01.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:01 smithi050 ceph-mon[30378]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:01.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:01 smithi050 ceph-mon[30378]: from='osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:20:01.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:01 smithi050 ceph-mon[30378]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:20:02.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:01 smithi142 ceph-mon[35329]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:02.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:01 smithi142 ceph-mon[35329]: from='osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:20:02.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:01 smithi142 ceph-mon[35329]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:20:02.815 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:02 smithi082 ceph-mon[35046]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:20:02.815 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:02 smithi082 ceph-mon[35046]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:20:02.816 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:02 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:02.816 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:02 smithi082 ceph-mon[35046]: from='osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:20:02.816 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:02 smithi082 ceph-mon[35046]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:20:02.817 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:20:02.842 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-09-12T03:20:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:02 smithi050 ceph-mon[30378]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:20:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:02 smithi050 ceph-mon[30378]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:20:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:02 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:02 smithi050 ceph-mon[30378]: from='osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:20:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:02 smithi050 ceph-mon[30378]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:20:03.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:02 smithi142 ceph-mon[35329]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:20:03.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:02 smithi142 ceph-mon[35329]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:20:03.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:02 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:02 smithi142 ceph-mon[35329]: from='osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:20:03.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:02 smithi142 ceph-mon[35329]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:20:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 -- 172.21.15.82:0/1242385974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 msgr2=0x7ff58c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 --2- 172.21.15.82:0/1242385974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c10a050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff584009900 tx=0x7ff58402a410 comp rx=0 tx=0).stop 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 -- 172.21.15.82:0/1242385974 shutdown_connections 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 --2- 172.21.15.82:0/1242385974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c10a050 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 --2- 172.21.15.82:0/1242385974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c102e00 0x7ff58c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 --2- 172.21.15.82:0/1242385974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 0x7ff58c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 -- 172.21.15.82:0/1242385974 >> 172.21.15.82:0/1242385974 conn(0x7ff58c0fd390 msgr2=0x7ff58c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 -- 172.21.15.82:0/1242385974 shutdown_connections 2024-09-12T03:20:03.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 -- 172.21.15.82:0/1242385974 wait complete. 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 Processor -- start 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.550+0000 7ff5938be640 1 -- start start 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 0x7ff58c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c102e00 0x7ff58c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c10d880 con 0x7ff58c101c00 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c10d700 con 0x7ff58c1037c0 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c10da00 con 0x7ff58c102e00 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47234/0 (socket says 172.21.15.82:47234) 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 -- 172.21.15.82:0/1328877816 learned_addr learned my addr 172.21.15.82:0/1328877816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff59165a640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 0x7ff58c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff590e59640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c102e00 0x7ff58c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 -- 172.21.15.82:0/1328877816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c102e00 msgr2=0x7ff58c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c102e00 0x7ff58c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 -- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 msgr2=0x7ff58c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 0x7ff58c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 -- 172.21.15.82:0/1328877816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0753e0 con 0x7ff58c1037c0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff59165a640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 0x7ff58c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff591e5b640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c074b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5840314e0 tx=0x7ff5840043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58403f030 con 0x7ff58c1037c0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff584003ce0 con 0x7ff58c1037c0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1a1f50 con 0x7ff58c1037c0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.551+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584035070 con 0x7ff58c1037c0 2024-09-12T03:20:03.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.552+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a23a0 con 0x7ff58c1037c0 2024-09-12T03:20:03.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.553+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c108f30 con 0x7ff58c1037c0 2024-09-12T03:20:03.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.554+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff584042030 con 0x7ff58c1037c0 2024-09-12T03:20:03.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.554+0000 7ff5827fc640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff5780a2b80 0x7ff5780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.555+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7ff5840eeb10 con 0x7ff58c1037c0 2024-09-12T03:20:03.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.555+0000 7ff59165a640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff5780a2b80 0x7ff5780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.555+0000 7ff59165a640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff5780a2b80 0x7ff5780a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff58c102c60 tx=0x7ff588007870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:03.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.558+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5840bcc70 con 0x7ff58c1037c0 2024-09-12T03:20:03.634 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3[50827]: 2024-09-12T03:20:03.242+0000 7f786ac00640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T03:20:03.635 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3[50827]: 2024-09-12T03:20:03.258+0000 7f7860200640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:20:03.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:03.736+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff58c062630 con 0x7ff5780a2b80 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='osd.3 ' entity='osd.3' 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] boot 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.737 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:03.738 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:03.738 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.738 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:03.738 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:03.738 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:20:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T03:20:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='osd.3 ' entity='osd.3' 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] boot 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:03.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:04.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='osd.3 ' entity='osd.3' 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: osd.3 [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] boot 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:04.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:04.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:04.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:04.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:04.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:04.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:04.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi050 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi050 to 8547M 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:04.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:04 smithi082 ceph-mon[35046]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:20:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:20:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:20:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi050 2024-09-12T03:20:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi050 to 8547M 2024-09-12T03:20:04.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:04.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:04.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:04.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:04 smithi050 ceph-mon[30378]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:20:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:20:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:20:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi050 2024-09-12T03:20:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi050 to 8547M 2024-09-12T03:20:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:04 smithi142 ceph-mon[35329]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:20:05.720 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:05 smithi082 ceph-mon[35046]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:05.720 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:05 smithi082 ceph-mon[35046]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:05.720 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:05 smithi082 ceph-mon[35046]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T03:20:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:05 smithi050 ceph-mon[30378]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:05 smithi050 ceph-mon[30378]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:05 smithi050 ceph-mon[30378]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T03:20:06.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:05 smithi142 ceph-mon[35329]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:06.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:05 smithi142 ceph-mon[35329]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:06.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:05 smithi142 ceph-mon[35329]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T03:20:06.796 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:06 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/1678897152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]: dispatch 2024-09-12T03:20:06.796 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:06 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]: dispatch 2024-09-12T03:20:06.796 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:06 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]': finished 2024-09-12T03:20:06.796 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:06 smithi082 ceph-mon[35046]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:20:06.796 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:06 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/1678897152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]: dispatch 2024-09-12T03:20:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:06 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]: dispatch 2024-09-12T03:20:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:06 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]': finished 2024-09-12T03:20:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:06 smithi050 ceph-mon[30378]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:20:06.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:06 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/1678897152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]: dispatch 2024-09-12T03:20:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:06 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]: dispatch 2024-09-12T03:20:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:06 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61ff9884-cde7-4179-836b-0acc8b7a0e2b"}]': finished 2024-09-12T03:20:07.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:06 smithi142 ceph-mon[35329]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:20:07.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:07.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:07 smithi050 ceph-mon[30378]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:07.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:07 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/957390472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:07.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:07 smithi050 ceph-mon[30378]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:20:07.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:08.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:07 smithi142 ceph-mon[35329]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:08.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:07 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/957390472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:08.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:07 smithi142 ceph-mon[35329]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:20:08.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:08.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:07 smithi082 ceph-mon[35046]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:08.207 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:07 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/957390472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:08.208 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:07 smithi082 ceph-mon[35046]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:20:08.208 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:08 smithi142 ceph-mon[35329]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:08 smithi082 ceph-mon[35046]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:09.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:08 smithi050 ceph-mon[30378]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:11.054 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:10 smithi082 ceph-mon[35046]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:20:11.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:10 smithi050 ceph-mon[30378]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:20:11.249 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:10 smithi142 ceph-mon[35329]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:20:12.828 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:12 smithi082 ceph-mon[35046]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:20:13.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:12 smithi142 ceph-mon[35329]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:20:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:12 smithi050 ceph-mon[30378]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:20:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:14 smithi142 ceph-mon[35329]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:20:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:14 smithi082 ceph-mon[35046]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:20:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:14 smithi050 ceph-mon[30378]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:20:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:20:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:15 smithi082 ceph-mon[35046]: Deploying daemon osd.4 on smithi082 2024-09-12T03:20:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:20:16.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:16.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:15 smithi142 ceph-mon[35329]: Deploying daemon osd.4 on smithi082 2024-09-12T03:20:16.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:20:16.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:16.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:15 smithi050 ceph-mon[30378]: Deploying daemon osd.4 on smithi082 2024-09-12T03:20:16.862 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:16 smithi082 ceph-mon[35046]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:20:17.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:16 smithi142 ceph-mon[35329]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:20:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:16 smithi050 ceph-mon[30378]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:20:19.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:18 smithi142 ceph-mon[35329]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:20:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:18 smithi082 ceph-mon[35046]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:20:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:18 smithi050 ceph-mon[30378]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:20:21.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:20 smithi142 ceph-mon[35329]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:20:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:20 smithi082 ceph-mon[35046]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:20:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:20 smithi050 ceph-mon[30378]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:20:22.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:22.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:22.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:22 smithi142 ceph-mon[35329]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:22 smithi082 ceph-mon[35046]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:22 smithi050 ceph-mon[30378]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:24.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.302+0000 7ff5827fc640 1 -- 172.21.15.82:0/1328877816 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff58c062630 con 0x7ff5780a2b80 2024-09-12T03:20:24.303 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 4 on host 'smithi082' 2024-09-12T03:20:24.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff5780a2b80 msgr2=0x7ff5780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff5780a2b80 0x7ff5780a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff58c102c60 tx=0x7ff588007870 comp rx=0 tx=0).stop 2024-09-12T03:20:24.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 msgr2=0x7ff58c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c074b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5840314e0 tx=0x7ff5840043c0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 shutdown_connections 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff5780a2b80 0x7ff5780a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58c1037c0 0x7ff58c074b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c102e00 0x7ff58c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 --2- 172.21.15.82:0/1328877816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff58c101c00 0x7ff58c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 >> 172.21.15.82:0/1328877816 conn(0x7ff58c0fd390 msgr2=0x7ff58c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 shutdown_connections 2024-09-12T03:20:24.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:20:24.305+0000 7ff5938be640 1 -- 172.21.15.82:0/1328877816 wait complete. 2024-09-12T03:20:24.912 DEBUG:teuthology.orchestra.run.smithi082:osd.4> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T03:20:24.915 INFO:tasks.cephadm:Deploying osd.5 on smithi142 with /dev/vg_nvme/lv_3... 2024-09-12T03:20:24.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:20:25.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:25.134 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:25.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.135 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:24 smithi082 ceph-mon[35046]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:24 smithi050 ceph-mon[30378]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:25.166 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:20:25.287 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.287 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.287 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:25.288 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:25.288 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.288 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.288 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.288 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:25.288 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:24 smithi142 ceph-mon[35329]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:26.953 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:26 smithi142 ceph-mon[35329]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:27.071 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:26 smithi050 ceph-mon[30378]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:26 smithi082 ceph-mon[35046]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:27.414 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:20:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:20:27.330+0000 7f2933107640 -1 osd.4 0 log_to_monitors true 2024-09-12T03:20:27.727 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:20:27.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-09-12T03:20:28.003 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:20:28.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:27 smithi142 ceph-mon[35329]: from='osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:20:28.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:27 smithi142 ceph-mon[35329]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:20:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:27 smithi082 ceph-mon[35046]: from='osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:20:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:27 smithi082 ceph-mon[35046]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:20:28.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:27 smithi050 ceph-mon[30378]: from='osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:20:28.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:27 smithi050 ceph-mon[30378]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 -- 172.21.15.142:0/3166946469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 msgr2=0x7f75d4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 --2- 172.21.15.142:0/3166946469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4102150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009900 tx=0x7f75bc02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 -- 172.21.15.142:0/3166946469 shutdown_connections 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 --2- 172.21.15.142:0/3166946469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75d4102690 0x7f75d4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 --2- 172.21.15.142:0/3166946469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4102150 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 --2- 172.21.15.142:0/3166946469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 0x7f75d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 -- 172.21.15.142:0/3166946469 >> 172.21.15.142:0/3166946469 conn(0x7f75d40fc280 msgr2=0x7f75d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 -- 172.21.15.142:0/3166946469 shutdown_connections 2024-09-12T03:20:28.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.502+0000 7f75da16d640 1 -- 172.21.15.142:0/3166946469 wait complete. 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 Processor -- start 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 -- start start 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 0x7f75d4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75d4102690 0x7f75d419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75d410d8c0 con 0x7f75d4102690 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75d410d740 con 0x7f75d4100ad0 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75da16d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75d410da40 con 0x7f75d4101cd0 2024-09-12T03:20:28.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 0x7f75d4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44560/0 (socket says 172.21.15.142:44560) 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 -- 172.21.15.142:0/3852436530 learned_addr learned my addr 172.21.15.142:0/3852436530 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 0x7f75d4197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.142:37578/0 (socket says 172.21.15.142:37578) 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 -- 172.21.15.142:0/3852436530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 msgr2=0x7f75d4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 0x7f75d4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 -- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75d4102690 msgr2=0x7f75d419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75d4102690 0x7f75d419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.503+0000 7f75cbfff640 1 -- 172.21.15.142:0/3852436530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d419fb70 con 0x7f75d4101cd0 2024-09-12T03:20:28.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.504+0000 7f75cbfff640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4198480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009230 tx=0x7f75bc031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:28.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.504+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc03f030 con 0x7f75d4101cd0 2024-09-12T03:20:28.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.504+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d419fe00 con 0x7f75d4101cd0 2024-09-12T03:20:28.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.504+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc003ad0 con 0x7f75d4101cd0 2024-09-12T03:20:28.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.504+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc035070 con 0x7f75d4101cd0 2024-09-12T03:20:28.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.504+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a02d0 con 0x7f75d4101cd0 2024-09-12T03:20:28.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.505+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d4107e70 con 0x7f75d4101cd0 2024-09-12T03:20:28.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.506+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75bc042030 con 0x7f75d4101cd0 2024-09-12T03:20:28.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.507+0000 7f75d17fa640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f75c00a2bb0 0x7f75c00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.507+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3805+0+0 (secure 0 0 0) 0x7f75bc0ee7b0 con 0x7f75d4101cd0 2024-09-12T03:20:28.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.507+0000 7f75d37fe640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f75c00a2bb0 0x7f75c00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.508+0000 7f75d37fe640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f75c00a2bb0 0x7f75c00a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f75d4101b30 tx=0x7f75c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:28.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.510+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75bc0bc8b0 con 0x7f75d4101cd0 2024-09-12T03:20:28.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:28.684+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f75d4062630 con 0x7f75c00a2bb0 2024-09-12T03:20:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi082 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi082 to 8544M 2024-09-12T03:20:29.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:29.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:29.104 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:20:28.768+0000 7f292f08a640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T03:20:29.105 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:20:28.774+0000 7f292aeb2640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:20:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:20:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:20:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:20:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:20:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi082 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi082 to 8544M 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:29.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:29.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:29.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:29.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi082 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi082 to 8544M 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:29.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:29.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:29.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:29.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:30.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:29 smithi142 ceph-mon[35329]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:20:30.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:29 smithi142 ceph-mon[35329]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:20:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:29 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:29 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:29 smithi082 ceph-mon[35046]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:20:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:29 smithi082 ceph-mon[35046]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:20:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:29 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:29 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:29 smithi050 ceph-mon[30378]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:20:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:29 smithi050 ceph-mon[30378]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:20:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:30 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:20:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:30 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:20:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:30 smithi142 ceph-mon[35329]: osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] boot 2024-09-12T03:20:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:30 smithi142 ceph-mon[35329]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:20:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:30 smithi142 ceph-mon[35329]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:30 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:20:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:30 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:20:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:30 smithi082 ceph-mon[35046]: osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] boot 2024-09-12T03:20:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:30 smithi082 ceph-mon[35046]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:20:31.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:31.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:30 smithi082 ceph-mon[35046]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:30 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:20:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:30 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:20:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:30 smithi050 ceph-mon[30378]: osd.4 [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] boot 2024-09-12T03:20:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:30 smithi050 ceph-mon[30378]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:20:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:20:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:30 smithi050 ceph-mon[30378]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:20:32.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T03:20:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/4142152267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]: dispatch 2024-09-12T03:20:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]: dispatch 2024-09-12T03:20:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]': finished 2024-09-12T03:20:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:20:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:31 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/577601761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:32.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T03:20:32.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/4142152267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]: dispatch 2024-09-12T03:20:32.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]: dispatch 2024-09-12T03:20:32.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]': finished 2024-09-12T03:20:32.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:20:32.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:32.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:31 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/577601761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:32.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T03:20:32.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/4142152267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]: dispatch 2024-09-12T03:20:32.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]: dispatch 2024-09-12T03:20:32.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0636aa4c-98aa-4b31-86ea-d77fc89be048"}]': finished 2024-09-12T03:20:32.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:20:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:31 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/577601761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:32 smithi142 ceph-mon[35329]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:32 smithi082 ceph-mon[35046]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:32 smithi050 ceph-mon[30378]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:34 smithi142 ceph-mon[35329]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:35.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:34 smithi082 ceph-mon[35046]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:34 smithi050 ceph-mon[30378]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:36 smithi142 ceph-mon[35329]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:36 smithi082 ceph-mon[35046]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:36 smithi050 ceph-mon[30378]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:38 smithi142 ceph-mon[35329]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:38 smithi082 ceph-mon[35046]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:38 smithi050 ceph-mon[30378]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:39.990 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:20:39.990 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:39.990 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:39 smithi142 ceph-mon[35329]: Deploying daemon osd.5 on smithi142 2024-09-12T03:20:40.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:20:40.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:40.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:39 smithi082 ceph-mon[35046]: Deploying daemon osd.5 on smithi142 2024-09-12T03:20:40.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:20:40.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:40.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:39 smithi050 ceph-mon[30378]: Deploying daemon osd.5 on smithi142 2024-09-12T03:20:40.966 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:40 smithi142 ceph-mon[35329]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:40 smithi082 ceph-mon[35046]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:40 smithi050 ceph-mon[30378]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:42.872 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:42 smithi142 ceph-mon[35329]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:42 smithi082 ceph-mon[35046]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:42 smithi050 ceph-mon[30378]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:44.965 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:44 smithi142 ceph-mon[35329]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:44 smithi082 ceph-mon[35046]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:44 smithi050 ceph-mon[30378]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:46.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:45 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:45 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:45 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:46.065 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:45 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:46.065 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:45 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:46.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:45 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:45 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:45 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:45 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:47.017 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:46 smithi142 ceph-mon[35329]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:46 smithi082 ceph-mon[35046]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:47.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:46 smithi050 ceph-mon[30378]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:48.910 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:48 smithi142 ceph-mon[35329]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:48.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.949+0000 7f75d17fa640 1 -- 172.21.15.142:0/3852436530 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f75d4062630 con 0x7f75c00a2bb0 2024-09-12T03:20:48.949 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 5 on host 'smithi142' 2024-09-12T03:20:48.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f75c00a2bb0 msgr2=0x7f75c00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f75c00a2bb0 0x7f75c00a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f75d4101b30 tx=0x7f75c4005f90 comp rx=0 tx=0).stop 2024-09-12T03:20:48.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 msgr2=0x7f75d4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4198480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009230 tx=0x7f75bc031b60 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 shutdown_connections 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f75c00a2bb0 0x7f75c00a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75d4102690 0x7f75d419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d4101cd0 0x7f75d4198480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 --2- 172.21.15.142:0/3852436530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d4100ad0 0x7f75d4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 >> 172.21.15.142:0/3852436530 conn(0x7f75d40fc280 msgr2=0x7f75d4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 shutdown_connections 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:20:48.951+0000 7f75da16d640 1 -- 172.21.15.142:0/3852436530 wait complete. 2024-09-12T03:20:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:48 smithi082 ceph-mon[35046]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:48 smithi050 ceph-mon[30378]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:49.674 DEBUG:teuthology.orchestra.run.smithi142:osd.5> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T03:20:49.676 INFO:tasks.cephadm:Deploying osd.6 on smithi050 with /dev/vg_nvme/lv_2... 2024-09-12T03:20:49.676 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:20:50.059 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:50.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:49 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:49 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:49 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:49 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.200 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.200 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.200 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.201 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:49 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:49 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:49 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:49 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:49 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:50 smithi050 ceph-mon[30378]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:51.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:50 smithi142 ceph-mon[35329]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:51.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:50 smithi082 ceph-mon[35046]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:51.543 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:20:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:20:51.180+0000 7fe6bd10f640 -1 osd.5 0 log_to_monitors true 2024-09-12T03:20:52.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:51 smithi050 ceph-mon[30378]: from='osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:20:52.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:51 smithi050 ceph-mon[30378]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:20:52.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:51 smithi142 ceph-mon[35329]: from='osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:20:52.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:51 smithi142 ceph-mon[35329]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:20:52.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:51 smithi082 ceph-mon[35046]: from='osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:20:52.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:51 smithi082 ceph-mon[35046]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:20:52.993 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:20:53.024 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-09-12T03:20:53.297 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:20:52.973+0000 7fe6b9092640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T03:20:53.297 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:20:52.978+0000 7fe6b4eba640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:20:53.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:20:53.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:20:53.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:53.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:20:53.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi142 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:53.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:53.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:52 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:20:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:20:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:20:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:20:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi142 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:53.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:52 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.393 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:53.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi142 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:53.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:20:53.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:53.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:53.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:53.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:52 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:20:53.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 -- 172.21.15.50:0/3785392417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c104f60 msgr2=0x7fa08c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 --2- 172.21.15.50:0/3785392417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c104f60 0x7fa08c1053e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa078009900 tx=0x7fa07802a400 comp rx=0 tx=0).stop 2024-09-12T03:20:53.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 -- 172.21.15.50:0/3785392417 shutdown_connections 2024-09-12T03:20:53.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 --2- 172.21.15.50:0/3785392417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c105920 0x7fa08c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 --2- 172.21.15.50:0/3785392417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c104f60 0x7fa08c1053e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 --2- 172.21.15.50:0/3785392417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 0x7fa08c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 -- 172.21.15.50:0/3785392417 >> 172.21.15.50:0/3785392417 conn(0x7fa08c0ff5b0 msgr2=0x7fa08c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 -- 172.21.15.50:0/3785392417 shutdown_connections 2024-09-12T03:20:53.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.939+0000 7fa093800640 1 -- 172.21.15.50:0/3785392417 wait complete. 2024-09-12T03:20:53.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.940+0000 7fa093800640 1 Processor -- start 2024-09-12T03:20:53.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.940+0000 7fa093800640 1 -- start start 2024-09-12T03:20:53.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa093800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 0x7fa08c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa093800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c104f60 0x7fa08c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa093800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 0x7fa08c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa093800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10fb00 con 0x7fa08c105920 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa093800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10f980 con 0x7fa08c103e00 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa093800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10fc80 con 0x7fa08c104f60 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 0x7fa08c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 0x7fa08c1a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55790/0 (socket says 172.21.15.50:55790) 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 -- 172.21.15.50:0/321432595 learned_addr learned my addr 172.21.15.50:0/321432595 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa08be00640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c104f60 0x7fa08c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa090e00640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 0x7fa08c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 -- 172.21.15.50:0/321432595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c104f60 msgr2=0x7fa08c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c104f60 0x7fa08c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 -- 172.21.15.50:0/321432595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 msgr2=0x7fa08c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 0x7fa08c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa091800640 1 -- 172.21.15.50:0/321432595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c1a1d60 con 0x7fa08c105920 2024-09-12T03:20:53.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.941+0000 7fa090e00640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 0x7fa08c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:53.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.942+0000 7fa091800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 0x7fa08c1a16f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa08400eaa0 tx=0x7fa08400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.942+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08400cdb0 con 0x7fa08c105920 2024-09-12T03:20:53.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.942+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa084005020 con 0x7fa08c105920 2024-09-12T03:20:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.942+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c1a1ff0 con 0x7fa08c105920 2024-09-12T03:20:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.942+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0840116b0 con 0x7fa08c105920 2024-09-12T03:20:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.942+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c1a2550 con 0x7fa08c105920 2024-09-12T03:20:53.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.943+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c10b120 con 0x7fa08c105920 2024-09-12T03:20:53.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.945+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa084005560 con 0x7fa08c105920 2024-09-12T03:20:53.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.947+0000 7fa089600640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa0740a2c30 0x7fa0740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.947+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7fa0840ce690 con 0x7fa08c105920 2024-09-12T03:20:53.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.947+0000 7fa090e00640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa0740a2c30 0x7fa0740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.948+0000 7fa090e00640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa0740a2c30 0x7fa0740a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa080009fc0 tx=0x7fa080005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:53.950+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa084016030 con 0x7fa08c105920 2024-09-12T03:20:54.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:20:54.146+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa08c062630 con 0x7fa0740a2c30 2024-09-12T03:20:54.147 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:20:54.147 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T03:20:54.148 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.148 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.148 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] boot 2024-09-12T03:20:54.148 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:20:54.148 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:53 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] boot 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:20:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:53 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:20:54.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T03:20:54.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:54.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: osd.5 [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] boot 2024-09-12T03:20:54.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:20:54.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:53 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:20:55.085 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:20:55.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:54 smithi050 ceph-mon[30378]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:20:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:20:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:20:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:54 smithi142 ceph-mon[35329]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:20:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:20:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:20:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:20:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:55.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:20:55.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:54 smithi082 ceph-mon[35046]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:56 smithi142 ceph-mon[35329]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T03:20:56.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:56 smithi082 ceph-mon[35046]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T03:20:56.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:56 smithi050 ceph-mon[30378]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T03:20:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:57 smithi050 ceph-mon[30378]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T03:20:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:57 smithi050 ceph-mon[30378]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:57 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4264096420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd0f4cc5-c370-4f89-8332-b28096839552"}]: dispatch 2024-09-12T03:20:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:57 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4264096420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd0f4cc5-c370-4f89-8332-b28096839552"}]': finished 2024-09-12T03:20:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:57 smithi050 ceph-mon[30378]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:20:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:57 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:20:57.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:57 smithi142 ceph-mon[35329]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T03:20:57.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:57 smithi142 ceph-mon[35329]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:57.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:57 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4264096420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd0f4cc5-c370-4f89-8332-b28096839552"}]: dispatch 2024-09-12T03:20:57.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:57 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4264096420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd0f4cc5-c370-4f89-8332-b28096839552"}]': finished 2024-09-12T03:20:57.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:57 smithi142 ceph-mon[35329]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:20:57.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:57 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:20:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:57 smithi082 ceph-mon[35046]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T03:20:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:57 smithi082 ceph-mon[35046]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:57 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4264096420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd0f4cc5-c370-4f89-8332-b28096839552"}]: dispatch 2024-09-12T03:20:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:57 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4264096420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd0f4cc5-c370-4f89-8332-b28096839552"}]': finished 2024-09-12T03:20:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:57 smithi082 ceph-mon[35046]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:20:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:57 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:20:58.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/577780696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:58 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/577780696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:58.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:58 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/577780696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:20:59.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:20:59 smithi142 ceph-mon[35329]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:59.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:20:59 smithi082 ceph-mon[35046]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:20:59.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:20:59 smithi050 ceph-mon[30378]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:00 smithi142 ceph-mon[35329]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:21:01.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:00 smithi082 ceph-mon[35046]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:21:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:00 smithi050 ceph-mon[30378]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:21:02.892 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:02 smithi050 ceph-mon[30378]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:21:03.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:02 smithi142 ceph-mon[35329]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:21:03.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:02 smithi082 ceph-mon[35046]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:21:04.859 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:04 smithi050 ceph-mon[30378]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:21:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:04 smithi142 ceph-mon[35329]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:21:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:04 smithi082 ceph-mon[35046]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:21:06.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:21:06.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:06.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:21:06.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:06.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:21:06.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:07 smithi050 ceph-mon[30378]: Deploying daemon osd.6 on smithi050 2024-09-12T03:21:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:07 smithi050 ceph-mon[30378]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:21:07.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:07 smithi082 ceph-mon[35046]: Deploying daemon osd.6 on smithi050 2024-09-12T03:21:07.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:07 smithi082 ceph-mon[35046]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:21:07.528 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:07 smithi142 ceph-mon[35329]: Deploying daemon osd.6 on smithi050 2024-09-12T03:21:07.528 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:07 smithi142 ceph-mon[35329]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:21:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:08 smithi142 ceph-mon[35329]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T03:21:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:08 smithi082 ceph-mon[35046]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T03:21:09.138 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:08 smithi050 ceph-mon[30378]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T03:21:10.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:10 smithi050 ceph-mon[30378]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:21:11.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:10 smithi142 ceph-mon[35329]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:21:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:10 smithi082 ceph-mon[35046]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:21:13.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:12 smithi142 ceph-mon[35329]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:12 smithi082 ceph-mon[35046]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:13.143 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:12 smithi050 ceph-mon[30378]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:14.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:14.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:14.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:14.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:14.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:14.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:14 smithi050 ceph-mon[30378]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:14 smithi142 ceph-mon[35329]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:14 smithi082 ceph-mon[35046]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:17.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:16 smithi142 ceph-mon[35329]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:16 smithi082 ceph-mon[35046]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:17.107 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:16 smithi050 ceph-mon[30378]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:17.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.883+0000 7fa089600640 1 -- 172.21.15.50:0/321432595 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa08c062630 con 0x7fa0740a2c30 2024-09-12T03:21:17.886 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 6 on host 'smithi050' 2024-09-12T03:21:17.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa0740a2c30 msgr2=0x7fa0740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa0740a2c30 0x7fa0740a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa080009fc0 tx=0x7fa080005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:17.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 msgr2=0x7fa08c1a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:17.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 0x7fa08c1a16f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa08400eaa0 tx=0x7fa08400ef60 comp rx=0 tx=0).stop 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 shutdown_connections 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa0740a2c30 0x7fa0740a50f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa08c105920 0x7fa08c1a16f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c104f60 0x7fa08c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 --2- 172.21.15.50:0/321432595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08c103e00 0x7fa08c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.887+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 >> 172.21.15.50:0/321432595 conn(0x7fa08c0ff5b0 msgr2=0x7fa08c1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.888+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 shutdown_connections 2024-09-12T03:21:17.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:21:17.888+0000 7fa093800640 1 -- 172.21.15.50:0/321432595 wait complete. 2024-09-12T03:21:18.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:17 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:17 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:18.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:18.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:17 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.692 DEBUG:teuthology.orchestra.run.smithi050:osd.6> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T03:21:18.696 INFO:tasks.cephadm:Deploying osd.7 on smithi082 with /dev/vg_nvme/lv_2... 2024-09-12T03:21:18.696 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:21:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:18 smithi050 ceph-mon[30378]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:18.950 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:21:19.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:18 smithi142 ceph-mon[35329]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:18 smithi082 ceph-mon[35046]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:19.639 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:21:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:21:19.298+0000 7f6cb1614640 -1 osd.6 0 log_to_monitors true 2024-09-12T03:21:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:19 smithi050 ceph-mon[30378]: from='osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:21:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:19 smithi050 ceph-mon[30378]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:21:20.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:19 smithi082 ceph-mon[35046]: from='osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:21:20.141 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:19 smithi082 ceph-mon[35046]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:21:20.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:19 smithi142 ceph-mon[35329]: from='osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:21:20.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:19 smithi142 ceph-mon[35329]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:21:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:20 smithi082 ceph-mon[35046]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:21:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:20 smithi082 ceph-mon[35046]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:21:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:20 smithi082 ceph-mon[35046]: from='osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:21:21.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:20 smithi082 ceph-mon[35046]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:21:21.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:20 smithi082 ceph-mon[35046]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:20 smithi050 ceph-mon[30378]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:21:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:20 smithi050 ceph-mon[30378]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:21:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:21.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:20 smithi050 ceph-mon[30378]: from='osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:21:21.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:20 smithi050 ceph-mon[30378]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:21:21.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:20 smithi050 ceph-mon[30378]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:20 smithi142 ceph-mon[35329]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:21:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:20 smithi142 ceph-mon[35329]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:21:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:21.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:20 smithi142 ceph-mon[35329]: from='osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:21:21.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:20 smithi142 ceph-mon[35329]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:21:21.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:20 smithi142 ceph-mon[35329]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:21.593 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:21:21.619 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-09-12T03:21:21.871 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:21:21.899 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:21:21.507+0000 7f6cacc00640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T03:21:21.899 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:21:21.530+0000 7f6ca2200640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:21:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-mon[30378]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:21:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-mon[30378]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T03:21:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:22.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:21 smithi050 ceph-mon[30378]: from='osd.6 ' entity='osd.6' 2024-09-12T03:21:22.236 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:21 smithi082 ceph-mon[35046]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:21:22.236 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:21 smithi082 ceph-mon[35046]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T03:21:22.236 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:22.236 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:22.236 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:21 smithi082 ceph-mon[35046]: from='osd.6 ' entity='osd.6' 2024-09-12T03:21:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:21 smithi142 ceph-mon[35329]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:21:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:21 smithi142 ceph-mon[35329]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T03:21:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:21 smithi142 ceph-mon[35329]: from='osd.6 ' entity='osd.6' 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 -- 172.21.15.82:0/3575755599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 msgr2=0x7fe0d8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/3575755599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 0x7fe0d8100ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00d910 tx=0x7fe0cc02c380 comp rx=0 tx=0).stop 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 -- 172.21.15.82:0/3575755599 shutdown_connections 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/3575755599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0d8102690 0x7fe0d8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/3575755599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8101cd0 0x7fe0d8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/3575755599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 0x7fe0d8100ed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 -- 172.21.15.82:0/3575755599 >> 172.21.15.82:0/3575755599 conn(0x7fe0d80fc280 msgr2=0x7fe0d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:22.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 -- 172.21.15.82:0/3575755599 shutdown_connections 2024-09-12T03:21:22.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.321+0000 7fe0dccc8640 1 -- 172.21.15.82:0/3575755599 wait complete. 2024-09-12T03:21:22.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 Processor -- start 2024-09-12T03:21:22.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 -- start start 2024-09-12T03:21:22.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 0x7fe0d819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:22.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0d8101cd0 0x7fe0d819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:22.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 0x7fe0d81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:22.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d810b6b0 con 0x7fe0d8100ad0 2024-09-12T03:21:22.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d810b530 con 0x7fe0d8102690 2024-09-12T03:21:22.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.322+0000 7fe0dccc8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d810b830 con 0x7fe0d8101cd0 2024-09-12T03:21:22.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 0x7fe0d81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:22.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 0x7fe0d81a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57322/0 (socket says 172.21.15.82:57322) 2024-09-12T03:21:22.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 -- 172.21.15.82:0/2542302334 learned_addr learned my addr 172.21.15.82:0/2542302334 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-09-12T03:21:22.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d5d9b640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0d8101cd0 0x7fe0d819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:22.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d659c640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 0x7fe0d819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 -- 172.21.15.82:0/2542302334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0d8101cd0 msgr2=0x7fe0d819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0d8101cd0 0x7fe0d819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 -- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 msgr2=0x7fe0d819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 0x7fe0d819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 -- 172.21.15.82:0/2542302334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d81a1cc0 con 0x7fe0d8102690 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0d6d9d640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 0x7fe0d81a1650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000cad0 tx=0x7fe0d000ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000ad60 con 0x7fe0d8102690 2024-09-12T03:21:22.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d0005020 con 0x7fe0d8102690 2024-09-12T03:21:22.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d81a1fb0 con 0x7fe0d8102690 2024-09-12T03:21:22.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000f6b0 con 0x7fe0d8102690 2024-09-12T03:21:22.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.323+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d81a24b0 con 0x7fe0d8102690 2024-09-12T03:21:22.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.324+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d8107e80 con 0x7fe0d8102690 2024-09-12T03:21:22.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.326+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe0d0005560 con 0x7fe0d8102690 2024-09-12T03:21:22.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.327+0000 7fe0c77fe640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe0c00a2c30 0x7fe0c00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:22.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.327+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7fe0d00ce570 con 0x7fe0d8102690 2024-09-12T03:21:22.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.327+0000 7fe0d659c640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe0c00a2c30 0x7fe0c00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:22.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.328+0000 7fe0d659c640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe0c00a2c30 0x7fe0c00a50f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0048b0 tx=0x7fe0cc005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:22.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.330+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0d0014030 con 0x7fe0d8102690 2024-09-12T03:21:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:22.512+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe0d8062630 con 0x7fe0c00a2c30 2024-09-12T03:21:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:21:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:21:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] boot 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:23.171 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:21:23.171 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:21:23.171 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:23.171 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:23.171 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] boot 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:23.172 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: osd.6 [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] boot 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:21:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi050 2024-09-12T03:21:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi050 to 5698M 2024-09-12T03:21:24.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:24.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:24.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.246 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi050 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi050 to 5698M 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:24.247 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi050 2024-09-12T03:21:24.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi050 to 5698M 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:24 smithi082 ceph-mon[35046]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T03:21:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:24 smithi082 ceph-mon[35046]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:24 smithi142 ceph-mon[35329]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T03:21:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:24 smithi142 ceph-mon[35329]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:24 smithi050 ceph-mon[30378]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T03:21:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:24 smithi050 ceph-mon[30378]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:26.195 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:25 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/3598774534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]: dispatch 2024-09-12T03:21:26.195 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:25 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]: dispatch 2024-09-12T03:21:26.195 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:25 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]': finished 2024-09-12T03:21:26.195 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:25 smithi142 ceph-mon[35329]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:21:26.195 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:25 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:26.196 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:25 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/3266289683' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:25 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/3598774534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]: dispatch 2024-09-12T03:21:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:25 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]: dispatch 2024-09-12T03:21:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:25 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]': finished 2024-09-12T03:21:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:25 smithi082 ceph-mon[35046]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:21:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:25 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:25 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/3266289683' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:26.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:25 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/3598774534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]: dispatch 2024-09-12T03:21:26.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:25 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]: dispatch 2024-09-12T03:21:26.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:25 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a73508ea-e10f-466a-8ce6-fcdffbb2f047"}]': finished 2024-09-12T03:21:26.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:25 smithi050 ceph-mon[30378]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:21:26.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:25 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:26.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:25 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/3266289683' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:27.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:26 smithi142 ceph-mon[35329]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:27.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:26 smithi082 ceph-mon[35046]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:27.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:26 smithi050 ceph-mon[30378]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:28 smithi142 ceph-mon[35329]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:28 smithi082 ceph-mon[35046]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:28 smithi050 ceph-mon[30378]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:30 smithi082 ceph-mon[35046]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:30 smithi142 ceph-mon[35329]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:30 smithi050 ceph-mon[30378]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:32.820 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:32 smithi082 ceph-mon[35046]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:32 smithi142 ceph-mon[35329]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:32 smithi050 ceph-mon[30378]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:33 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:21:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:33 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:33.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:33 smithi082 ceph-mon[35046]: Deploying daemon osd.7 on smithi082 2024-09-12T03:21:34.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:33 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:21:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:33 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:33 smithi142 ceph-mon[35329]: Deploying daemon osd.7 on smithi082 2024-09-12T03:21:34.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:33 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:21:34.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:33 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:34.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:33 smithi050 ceph-mon[30378]: Deploying daemon osd.7 on smithi082 2024-09-12T03:21:34.947 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:34 smithi082 ceph-mon[35046]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:34 smithi142 ceph-mon[35329]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:34 smithi050 ceph-mon[30378]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:36 smithi082 ceph-mon[35046]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:36 smithi142 ceph-mon[35329]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:36 smithi050 ceph-mon[30378]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:38 smithi082 ceph-mon[35046]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:38 smithi142 ceph-mon[35329]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:38 smithi050 ceph-mon[30378]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:40.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:40.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:40.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:40.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:40.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:40.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:40.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:40.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:40.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:40 smithi050 ceph-mon[30378]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:41.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:40 smithi142 ceph-mon[35329]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:41.349 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:40 smithi082 ceph-mon[35046]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:42 smithi142 ceph-mon[35329]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:43.088 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:42 smithi082 ceph-mon[35046]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:42 smithi050 ceph-mon[30378]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:43.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.410+0000 7fe0c77fe640 1 -- 172.21.15.82:0/2542302334 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe0d8062630 con 0x7fe0c00a2c30 2024-09-12T03:21:43.411 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 7 on host 'smithi082' 2024-09-12T03:21:43.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe0c00a2c30 msgr2=0x7fe0c00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe0c00a2c30 0x7fe0c00a50f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0048b0 tx=0x7fe0cc005c00 comp rx=0 tx=0).stop 2024-09-12T03:21:43.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 msgr2=0x7fe0d81a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 0x7fe0d81a1650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000cad0 tx=0x7fe0d000ced0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 shutdown_connections 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe0c00a2c30 0x7fe0c00a50f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0d8102690 0x7fe0d81a1650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0d8101cd0 0x7fe0d819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 --2- 172.21.15.82:0/2542302334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0d8100ad0 0x7fe0d819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 >> 172.21.15.82:0/2542302334 conn(0x7fe0d80fc280 msgr2=0x7fe0d8105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 shutdown_connections 2024-09-12T03:21:43.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:21:43.413+0000 7fe0dccc8640 1 -- 172.21.15.82:0/2542302334 wait complete. 2024-09-12T03:21:44.095 DEBUG:teuthology.orchestra.run.smithi082:osd.7> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T03:21:44.097 INFO:tasks.cephadm:Deploying osd.8 on smithi142 with /dev/vg_nvme/lv_2... 2024-09-12T03:21:44.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:21:44.353 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:21:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.677 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.677 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.677 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.677 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.704 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.704 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.704 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:44.704 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:45.541 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:45 smithi082 ceph-mon[35046]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:45.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:45 smithi050 ceph-mon[30378]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:45.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:45 smithi142 ceph-mon[35329]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:46.104 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:21:46 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:21:46.085+0000 7fa003f8c640 -1 osd.7 0 log_to_monitors true 2024-09-12T03:21:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:46 smithi142 ceph-mon[35329]: from='osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:21:46.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:46 smithi142 ceph-mon[35329]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:21:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:46 smithi082 ceph-mon[35046]: from='osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:21:46.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:46 smithi082 ceph-mon[35046]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:21:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:46 smithi050 ceph-mon[30378]: from='osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:21:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:46 smithi050 ceph-mon[30378]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:21:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:21:47.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:47.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:47 smithi142 ceph-mon[35329]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:21:47.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:21:47.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:21:47.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:21:47.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:47.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:47.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:47.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:47.606 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:47 smithi082 ceph-mon[35046]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:21:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:21:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:21:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:21:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:47.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:47 smithi050 ceph-mon[30378]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:21:48.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 -- 172.21.15.142:0/156377072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5741026a0 msgr2=0x7fd574104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 --2- 172.21.15.142:0/156377072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5741026a0 0x7fd574104a90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd570009900 tx=0x7fd57002a410 comp rx=0 tx=0).stop 2024-09-12T03:21:48.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 -- 172.21.15.142:0/156377072 shutdown_connections 2024-09-12T03:21:48.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 --2- 172.21.15.142:0/156377072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574107900 0x7fd574109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 --2- 172.21.15.142:0/156377072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd574104fd0 0x7fd5741073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 --2- 172.21.15.142:0/156377072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5741026a0 0x7fd574104a90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 -- 172.21.15.142:0/156377072 >> 172.21.15.142:0/156377072 conn(0x7fd5740fc2a0 msgr2=0x7fd5740fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 -- 172.21.15.142:0/156377072 shutdown_connections 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.066+0000 7fd57c071640 1 -- 172.21.15.142:0/156377072 wait complete. 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.067+0000 7fd57c071640 1 Processor -- start 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.067+0000 7fd57c071640 1 -- start start 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57c071640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5741026a0 0x7fd574102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57c071640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574104fd0 0x7fd574100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57c071640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 0x7fd574100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57c071640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd57410d990 con 0x7fd574104fd0 2024-09-12T03:21:48.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57c071640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd57410d810 con 0x7fd5741026a0 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57c071640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd57410db10 con 0x7fd574107900 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 0x7fd574100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd579e0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5741026a0 0x7fd574102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd579e0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5741026a0 0x7fd574102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.142:38522/0 (socket says 172.21.15.142:38522) 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 0x7fd574100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35410/0 (socket says 172.21.15.142:35410) 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd579e0d640 1 -- 172.21.15.142:0/2398461328 learned_addr learned my addr 172.21.15.142:0/2398461328 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57960c640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574104fd0 0x7fd574100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 -- 172.21.15.142:0/2398461328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5741026a0 msgr2=0x7fd574102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5741026a0 0x7fd574102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 -- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574104fd0 msgr2=0x7fd574100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574104fd0 0x7fd574100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 -- 172.21.15.142:0/2398461328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd574101450 con 0x7fd574107900 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57960c640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574104fd0 0x7fd574100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd57a60e640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 0x7fd574100b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00be20 tx=0x7fd56c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c019070 con 0x7fd574107900 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.068+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56c014920 con 0x7fd574107900 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.069+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741a2200 con 0x7fd574107900 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.069+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c012920 con 0x7fd574107900 2024-09-12T03:21:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.069+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5741a2740 con 0x7fd574107900 2024-09-12T03:21:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.070+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5740fdd90 con 0x7fd574107900 2024-09-12T03:21:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.071+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd56c01c070 con 0x7fd574107900 2024-09-12T03:21:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.072+0000 7fd56affd640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd5600a2b80 0x7fd5600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.072+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7fd56c0cdc00 con 0x7fd574107900 2024-09-12T03:21:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.072+0000 7fd579e0d640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd5600a2b80 0x7fd5600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.072+0000 7fd579e0d640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd5600a2b80 0x7fd5600a5040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd570031440 tx=0x7fd5700315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.076+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd56c0d4080 con 0x7fd574107900 2024-09-12T03:21:48.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:21:48.256+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd574062630 con 0x7fd5600a2b80 2024-09-12T03:21:48.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:21:48.087+0000 7f9ffff0f640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T03:21:48.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:21:48.095+0000 7f9ff7d2f640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:21:48.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi082 2024-09-12T03:21:48.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi082 to 5696M 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='osd.7 ' entity='osd.7' 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:48.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:48.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] boot 2024-09-12T03:21:48.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:21:48.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:48 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi082 2024-09-12T03:21:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi082 to 5696M 2024-09-12T03:21:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T03:21:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='osd.7 ' entity='osd.7' 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] boot 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:21:48.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:48 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi082 2024-09-12T03:21:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi082 to 5696M 2024-09-12T03:21:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T03:21:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='osd.7 ' entity='osd.7' 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: osd.7 [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] boot 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:21:48.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:48 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:21:49.623 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:49 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:21:49.623 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:49 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:21:49.624 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:49 smithi142 ceph-mon[35329]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:49.624 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:49 smithi142 ceph-mon[35329]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:21:49.624 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:49 smithi142 ceph-mon[35329]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:49.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:49 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:21:49.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:49 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:21:49.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:49 smithi082 ceph-mon[35046]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:49.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:49 smithi082 ceph-mon[35046]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:21:49.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:49 smithi082 ceph-mon[35046]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:49 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:21:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:49 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:21:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:49 smithi050 ceph-mon[30378]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:49 smithi050 ceph-mon[30378]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:21:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:49 smithi050 ceph-mon[30378]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:50.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:50 smithi142 ceph-mon[35329]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T03:21:50.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:50 smithi082 ceph-mon[35046]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T03:21:50.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:50 smithi050 ceph-mon[30378]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T03:21:51.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T03:21:51.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/2673042335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]: dispatch 2024-09-12T03:21:51.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]: dispatch 2024-09-12T03:21:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]': finished 2024-09-12T03:21:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:21:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:21:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:51 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/2460760189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/2673042335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]: dispatch 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]: dispatch 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]': finished 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:51 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/2460760189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:51.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T03:21:51.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/2673042335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]: dispatch 2024-09-12T03:21:51.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]: dispatch 2024-09-12T03:21:51.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe07bf7-a040-4ddc-b175-de41dae683c5"}]': finished 2024-09-12T03:21:51.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:21:51.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:21:51.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:51.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:51 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/2460760189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:53.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:52 smithi142 ceph-mon[35329]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:53.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:52 smithi082 ceph-mon[35046]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:52 smithi050 ceph-mon[30378]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:55.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:54 smithi142 ceph-mon[35329]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:54 smithi082 ceph-mon[35046]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:55.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:54 smithi050 ceph-mon[30378]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:57.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:56 smithi142 ceph-mon[35329]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:57.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:56 smithi082 ceph-mon[35046]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:56 smithi050 ceph-mon[30378]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:58 smithi142 ceph-mon[35329]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:59.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:58 smithi082 ceph-mon[35046]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:21:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:58 smithi050 ceph-mon[30378]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:00.023 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:22:00.023 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:59 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:00.024 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:21:59 smithi082 ceph-mon[35046]: Deploying daemon osd.8 on smithi142 2024-09-12T03:22:00.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:22:00.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:59 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:00.047 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:21:59 smithi142 ceph-mon[35329]: Deploying daemon osd.8 on smithi142 2024-09-12T03:22:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:22:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:59 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:21:59 smithi050 ceph-mon[30378]: Deploying daemon osd.8 on smithi142 2024-09-12T03:22:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:00 smithi142 ceph-mon[35329]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:01.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:00 smithi082 ceph-mon[35046]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:00 smithi050 ceph-mon[30378]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:03.018 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:02 smithi142 ceph-mon[35329]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:03.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:02 smithi082 ceph-mon[35046]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:02 smithi050 ceph-mon[30378]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:04 smithi142 ceph-mon[35329]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:04 smithi082 ceph-mon[35046]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:04 smithi050 ceph-mon[30378]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:06.036 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:05 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:06.036 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:05 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:06.036 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:05 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:06.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:05 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:06.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:05 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:06.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:05 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:06 smithi142 ceph-mon[35329]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:07.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:06 smithi082 ceph-mon[35046]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:06 smithi050 ceph-mon[30378]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:09.383 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 8 on host 'smithi142' 2024-09-12T03:22:09.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.381+0000 7fd56affd640 1 -- 172.21.15.142:0/2398461328 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd574062630 con 0x7fd5600a2b80 2024-09-12T03:22:09.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.383+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd5600a2b80 msgr2=0x7fd5600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd5600a2b80 0x7fd5600a5040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd570031440 tx=0x7fd5700315b0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 msgr2=0x7fd574100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 0x7fd574100b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00be20 tx=0x7fd56c00bf10 comp rx=0 tx=0).stop 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 shutdown_connections 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd5600a2b80 0x7fd5600a5040 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd574107900 0x7fd574100b90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd574104fd0 0x7fd574100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 --2- 172.21.15.142:0/2398461328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5741026a0 0x7fd574102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 >> 172.21.15.142:0/2398461328 conn(0x7fd5740fc2a0 msgr2=0x7fd5741030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 shutdown_connections 2024-09-12T03:22:09.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:22:09.384+0000 7fd57c071640 1 -- 172.21.15.142:0/2398461328 wait complete. 2024-09-12T03:22:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:09 smithi082 ceph-mon[35046]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:09 smithi050 ceph-mon[30378]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:09.694 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.694 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.694 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:09.694 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:09.694 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:09.694 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:09 smithi142 ceph-mon[35329]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:10.021 DEBUG:teuthology.orchestra.run.smithi142:osd.8> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T03:22:10.024 INFO:tasks.cephadm:Deploying osd.9 on smithi050 with /dev/vg_nvme/lv_1... 2024-09-12T03:22:10.024 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:22:10.408 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:10.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:10 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:10.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:10 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:10.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:10 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:10.506 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:10 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:10.506 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:10 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:10.507 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:10 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:10 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:10 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:10 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:11.294 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:22:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:22:11.012+0000 7fc8ecda9640 -1 osd.8 0 log_to_monitors true 2024-09-12T03:22:11.572 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:11 smithi142 ceph-mon[35329]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:11.572 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:11 smithi142 ceph-mon[35329]: from='osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:22:11.572 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:11 smithi142 ceph-mon[35329]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:22:11.590 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:11 smithi050 ceph-mon[30378]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:11.590 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:11 smithi050 ceph-mon[30378]: from='osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:22:11.590 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:11 smithi050 ceph-mon[30378]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:22:11.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:11 smithi082 ceph-mon[35046]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:11.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:11 smithi082 ceph-mon[35046]: from='osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:22:11.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:11 smithi082 ceph-mon[35046]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:22:12.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:12 smithi050 ceph-mon[30378]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:22:12.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:12 smithi050 ceph-mon[30378]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:22:12.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:12.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:12 smithi050 ceph-mon[30378]: from='osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:22:12.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:12 smithi050 ceph-mon[30378]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:22:12.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-mon[35329]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:22:12.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-mon[35329]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:22:12.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:12.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-mon[35329]: from='osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:22:12.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-mon[35329]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:22:12.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:12 smithi082 ceph-mon[35046]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:22:12.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:12 smithi082 ceph-mon[35046]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:22:12.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:12.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:12 smithi082 ceph-mon[35046]: from='osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:22:12.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:12 smithi082 ceph-mon[35046]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:22:13.293 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:22:12.983+0000 7fc8e8d2c640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T03:22:13.293 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:22:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:22:12.992+0000 7fc8e0b4c640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='osd.8 ' entity='osd.8' 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:13.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.576 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='osd.8 ' entity='osd.8' 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:13.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-09-12T03:22:13.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='osd.8 ' entity='osd.8' 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:13.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:13.979 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:14.389 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi142 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] boot 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:14.390 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:14 smithi050 ceph-mon[30378]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:22:14.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.506+0000 7efdb1200640 1 -- 172.21.15.50:0/2966376375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac076ba0 msgr2=0x7efdac076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.506+0000 7efdb1200640 1 --2- 172.21.15.50:0/2966376375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac076ba0 0x7efdac076fa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a410 comp rx=0 tx=0).stop 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 -- 172.21.15.50:0/2966376375 shutdown_connections 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 --2- 172.21.15.50:0/2966376375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdac078790 0x7efdac079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 --2- 172.21.15.50:0/2966376375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac077df0 0x7efdac078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 --2- 172.21.15.50:0/2966376375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac076ba0 0x7efdac076fa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 -- 172.21.15.50:0/2966376375 >> 172.21.15.50:0/2966376375 conn(0x7efdac0ff5b0 msgr2=0x7efdac1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 -- 172.21.15.50:0/2966376375 shutdown_connections 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.507+0000 7efdb1200640 1 -- 172.21.15.50:0/2966376375 wait complete. 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 Processor -- start 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 -- start start 2024-09-12T03:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac076ba0 0x7efdac19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 0x7efdac19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdac078790 0x7efdac1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdac113ea0 con 0x7efdac077df0 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efdac113d20 con 0x7efdac076ba0 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdb1200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efdac114020 con 0x7efdac078790 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efda1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 0x7efdac19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efda1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 0x7efdac19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49872/0 (socket says 172.21.15.50:49872) 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efda1400640 1 -- 172.21.15.50:0/632160029 learned_addr learned my addr 172.21.15.50:0/632160029 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efda9e00640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac076ba0 0x7efdac19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.508+0000 7efdaa800640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdac078790 0x7efdac1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda1400640 1 -- 172.21.15.50:0/632160029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdac078790 msgr2=0x7efdac1a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda1400640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdac078790 0x7efdac1a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda1400640 1 -- 172.21.15.50:0/632160029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac076ba0 msgr2=0x7efdac19e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda1400640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac076ba0 0x7efdac19e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda1400640 1 -- 172.21.15.50:0/632160029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac1a60d0 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda1400640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 0x7efdac19e9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd9400d950 tx=0x7efd9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94017030 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd94004020 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac1a63c0 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.509+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940047a0 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.510+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1a6900 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.511+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac110780 con 0x7efdac077df0 2024-09-12T03:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.513+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efd940041d0 con 0x7efdac077df0 2024-09-12T03:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.517+0000 7efda3400640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efd900a2bb0 0x7efd900a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.517+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5858+0+0 (secure 0 0 0) 0x7efd940ce770 con 0x7efdac077df0 2024-09-12T03:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.517+0000 7efda9e00640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efd900a2bb0 0x7efd900a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.517+0000 7efda9e00640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efd900a2bb0 0x7efd900a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0314e0 tx=0x7efd9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.518+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd9409bfc0 con 0x7efdac077df0 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi142 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] boot 2024-09-12T03:22:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:22:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:14 smithi082 ceph-mon[35046]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:22:14.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:14.712+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efdac062630 con 0x7efd900a2bb0 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi142 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: osd.8 [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] boot 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:22:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:22:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:14 smithi142 ceph-mon[35329]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:22:15.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:15 smithi050 ceph-mon[30378]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T03:22:15.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:15 smithi050 ceph-mon[30378]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:15.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:15.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:15.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:15 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:15.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:15 smithi142 ceph-mon[35329]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T03:22:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:15 smithi142 ceph-mon[35329]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:15.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:15 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:15 smithi082 ceph-mon[35046]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T03:22:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:15 smithi082 ceph-mon[35046]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:15 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:16 smithi142 ceph-mon[35329]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:16 smithi142 ceph-mon[35329]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T03:22:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:16 smithi082 ceph-mon[35046]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:16 smithi082 ceph-mon[35046]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T03:22:16.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:16 smithi050 ceph-mon[30378]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:16.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:16 smithi050 ceph-mon[30378]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T03:22:17.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:17 smithi142 ceph-mon[35329]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T03:22:17.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:17 smithi082 ceph-mon[35046]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T03:22:17.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:17 smithi050 ceph-mon[30378]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T03:22:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:18 smithi050 ceph-mon[30378]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:18 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2446836325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be1e8d37-b1a4-4f1f-a019-efa72dcaacaa"}]: dispatch 2024-09-12T03:22:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:18 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2446836325' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be1e8d37-b1a4-4f1f-a019-efa72dcaacaa"}]': finished 2024-09-12T03:22:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:18 smithi050 ceph-mon[30378]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:22:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:18.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:18 smithi142 ceph-mon[35329]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:18.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:18 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2446836325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be1e8d37-b1a4-4f1f-a019-efa72dcaacaa"}]: dispatch 2024-09-12T03:22:18.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:18 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2446836325' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be1e8d37-b1a4-4f1f-a019-efa72dcaacaa"}]': finished 2024-09-12T03:22:18.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:18 smithi142 ceph-mon[35329]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:22:18.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:18 smithi082 ceph-mon[35046]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:18 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2446836325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be1e8d37-b1a4-4f1f-a019-efa72dcaacaa"}]: dispatch 2024-09-12T03:22:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:18 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2446836325' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be1e8d37-b1a4-4f1f-a019-efa72dcaacaa"}]': finished 2024-09-12T03:22:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:18 smithi082 ceph-mon[35046]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:22:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:19.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:19 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3445431521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:19.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:19 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3445431521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:19 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3445431521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:20.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:20 smithi142 ceph-mon[35329]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-12T03:22:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:20 smithi082 ceph-mon[35046]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-12T03:22:20.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:20 smithi050 ceph-mon[30378]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-12T03:22:22.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:22 smithi142 ceph-mon[35329]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:22:22.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:22 smithi082 ceph-mon[35046]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:22:22.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:22 smithi050 ceph-mon[30378]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:22:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:24 smithi050 ceph-mon[30378]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:22:24.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:24 smithi142 ceph-mon[35329]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:22:24.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:24 smithi082 ceph-mon[35046]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:22:26.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:26 smithi142 ceph-mon[35329]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:22:26.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:26 smithi050 ceph-mon[30378]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:22:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:26 smithi082 ceph-mon[35046]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:22:28.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:28 smithi050 ceph-mon[30378]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:22:28.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:22:28.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:28.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:28 smithi050 ceph-mon[30378]: Deploying daemon osd.9 on smithi050 2024-09-12T03:22:28.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:28 smithi142 ceph-mon[35329]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:22:28.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:22:28.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:28.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:28 smithi142 ceph-mon[35329]: Deploying daemon osd.9 on smithi050 2024-09-12T03:22:28.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:28 smithi082 ceph-mon[35046]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:22:28.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:22:28.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:28.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:28 smithi082 ceph-mon[35046]: Deploying daemon osd.9 on smithi050 2024-09-12T03:22:30.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:30 smithi142 ceph-mon[35329]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T03:22:30.820 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:30 smithi050 ceph-mon[30378]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T03:22:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:30 smithi082 ceph-mon[35046]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T03:22:32.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:32 smithi142 ceph-mon[35329]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:32.807 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:32 smithi050 ceph-mon[30378]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:32 smithi082 ceph-mon[35046]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:34 smithi050 ceph-mon[30378]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:34.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:34 smithi142 ceph-mon[35329]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:34.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:34 smithi082 ceph-mon[35046]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:35.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:35.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:35.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:35.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:35.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:35.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:35.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:35.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:35.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:36.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:36 smithi050 ceph-mon[30378]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:36.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:36 smithi142 ceph-mon[35329]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:36 smithi082 ceph-mon[35046]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:38.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:38 smithi142 ceph-mon[35329]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:38.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:38.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:38.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:38 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:38 smithi082 ceph-mon[35046]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:38 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:38 smithi050 ceph-mon[30378]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:38.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:38.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:38.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:38 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.049+0000 7efda3400640 1 -- 172.21.15.50:0/632160029 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efdac062630 con 0x7efd900a2bb0 2024-09-12T03:22:39.050 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 9 on host 'smithi050' 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efd900a2bb0 msgr2=0x7efd900a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efd900a2bb0 0x7efd900a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0314e0 tx=0x7efd9c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 msgr2=0x7efdac19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 0x7efdac19e9e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd9400d950 tx=0x7efd9400de10 comp rx=0 tx=0).stop 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 shutdown_connections 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efd900a2bb0 0x7efd900a5070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdac078790 0x7efdac1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdac077df0 0x7efdac19e9e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 --2- 172.21.15.50:0/632160029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac076ba0 0x7efdac19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 >> 172.21.15.50:0/632160029 conn(0x7efdac0ff5b0 msgr2=0x7efdac101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 shutdown_connections 2024-09-12T03:22:39.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:22:39.051+0000 7efdb1200640 1 -- 172.21.15.50:0/632160029 wait complete. 2024-09-12T03:22:39.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:39.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:39 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.735 DEBUG:teuthology.orchestra.run.smithi050:osd.9> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T03:22:39.739 INFO:tasks.cephadm:Deploying osd.10 on smithi082 with /dev/vg_nvme/lv_1... 2024-09-12T03:22:39.739 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:22:39.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:39.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:39 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:39.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:39 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:39.995 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:22:40.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:40 smithi142 ceph-mon[35329]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:40.825 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:40 smithi050 ceph-mon[30378]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:40.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:40 smithi082 ceph-mon[35046]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:41.164 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:22:40 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:22:40.824+0000 7f8a68232640 -1 osd.9 0 log_to_monitors true 2024-09-12T03:22:41.663 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:41 smithi082 ceph-mon[35046]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:22:41.776 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:41 smithi050 ceph-mon[30378]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:22:41.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:41 smithi142 ceph-mon[35329]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:22:42.719 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:42 smithi082 ceph-mon[35046]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:42.719 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:42 smithi082 ceph-mon[35046]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:22:42.719 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:42 smithi082 ceph-mon[35046]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:22:42.719 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:42 smithi082 ceph-mon[35046]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:22:42.719 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:42 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:42.722 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-09-12T03:22:42.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-09-12T03:22:42.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:22:42.557+0000 7f8a62a00640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T03:22:42.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:22:42.572+0000 7f8a5da00640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:22:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-mon[30378]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-mon[30378]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:22:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-mon[30378]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:22:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-mon[30378]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:22:42.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:42 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:42.993 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:22:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:42 smithi142 ceph-mon[35329]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:42 smithi142 ceph-mon[35329]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:22:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:42 smithi142 ceph-mon[35329]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:22:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:42 smithi142 ceph-mon[35329]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-09-12T03:22:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:42 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:43.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 -- 172.21.15.82:0/1181265830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70100ad0 msgr2=0x7fea70100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 --2- 172.21.15.82:0/1181265830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70100ad0 0x7fea70100ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea60009900 tx=0x7fea6002a410 comp rx=0 tx=0).stop 2024-09-12T03:22:43.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 -- 172.21.15.82:0/1181265830 shutdown_connections 2024-09-12T03:22:43.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 --2- 172.21.15.82:0/1181265830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea70102690 0x7fea70108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 --2- 172.21.15.82:0/1181265830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea70101cd0 0x7fea70102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 --2- 172.21.15.82:0/1181265830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70100ad0 0x7fea70100ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 -- 172.21.15.82:0/1181265830 >> 172.21.15.82:0/1181265830 conn(0x7fea700fc280 msgr2=0x7fea700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 -- 172.21.15.82:0/1181265830 shutdown_connections 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.506+0000 7fea75e49640 1 -- 172.21.15.82:0/1181265830 wait complete. 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 Processor -- start 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 -- start start 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea70100ad0 0x7fea7010b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea70101cd0 0x7fea70109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 0x7fea70109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea7010d880 con 0x7fea70100ad0 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea7010d700 con 0x7fea70102690 2024-09-12T03:22:43.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea75e49640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea7010da00 con 0x7fea70101cd0 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 0x7fea70109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 0x7fea70109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49866/0 (socket says 172.21.15.82:49866) 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 -- 172.21.15.82:0/3883332006 learned_addr learned my addr 172.21.15.82:0/3883332006 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6f7fe640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea70100ad0 0x7fea7010b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea67fff640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea70101cd0 0x7fea70109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 -- 172.21.15.82:0/3883332006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea70101cd0 msgr2=0x7fea70109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea70101cd0 0x7fea70109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 -- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea70100ad0 msgr2=0x7fea7010b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.507+0000 7fea6ffff640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea70100ad0 0x7fea7010b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea6ffff640 1 -- 172.21.15.82:0/3883332006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7010a8a0 con 0x7fea70102690 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea6ffff640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 0x7fea70109fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fea6800eaa0 tx=0x7fea6800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800cdb0 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea701a20b0 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea68005020 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680116d0 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.508+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea701a2610 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.509+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea70100ed0 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.510+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea680054d0 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.510+0000 7fea6d7fa640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fea5c0a2b80 0x7fea5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.511+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5981+0+0 (secure 0 0 0) 0x7fea680ce8a0 con 0x7fea70102690 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.511+0000 7fea6f7fe640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fea5c0a2b80 0x7fea5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.511+0000 7fea6f7fe640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fea5c0a2b80 0x7fea5c0a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea60005ab0 tx=0x7fea60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.515+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea68016030 con 0x7fea70102690 2024-09-12T03:22:43.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:43.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:43 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:22:43.696+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fea70062630 con 0x7fea5c0a2b80 2024-09-12T03:22:43.697 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:22:43.697 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:43.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:43 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:44.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:44.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:43 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi050 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi050 to 4273M 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] boot 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T03:22:44.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:44.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:44.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:44.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:44.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi050 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi050 to 4273M 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] boot 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:44.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:44.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:44.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:44.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi050 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi050 to 4273M 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: osd.9 [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] boot 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:45.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:45 smithi082 ceph-mon[35046]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T03:22:45.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:45 smithi050 ceph-mon[30378]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T03:22:46.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:45 smithi142 ceph-mon[35329]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T03:22:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:46 smithi082 ceph-mon[35046]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:46 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/4172623698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]: dispatch 2024-09-12T03:22:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:46 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]: dispatch 2024-09-12T03:22:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:46 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]': finished 2024-09-12T03:22:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:46 smithi082 ceph-mon[35046]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:22:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:46 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:22:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:46 smithi050 ceph-mon[30378]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:46 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/4172623698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]: dispatch 2024-09-12T03:22:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:46 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]: dispatch 2024-09-12T03:22:46.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:46 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]': finished 2024-09-12T03:22:46.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:46 smithi050 ceph-mon[30378]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:22:46.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:22:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:46 smithi142 ceph-mon[35329]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:46 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/4172623698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]: dispatch 2024-09-12T03:22:47.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:46 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]: dispatch 2024-09-12T03:22:47.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:46 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2660198c-5848-4738-910b-711c291758a5"}]': finished 2024-09-12T03:22:47.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:46 smithi142 ceph-mon[35329]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:22:47.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:46 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:22:47.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:47 smithi082 ceph-mon[35046]: from='client.? 172.21.15.82:0/4042791709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:47.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:47 smithi050 ceph-mon[30378]: from='client.? 172.21.15.82:0/4042791709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:47 smithi142 ceph-mon[35329]: from='client.? 172.21.15.82:0/4042791709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:48 smithi082 ceph-mon[35046]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:48 smithi050 ceph-mon[30378]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:48 smithi142 ceph-mon[35329]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:50.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:50 smithi082 ceph-mon[35046]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:50.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:50 smithi050 ceph-mon[30378]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:51.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:50 smithi142 ceph-mon[35329]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:52 smithi082 ceph-mon[35046]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:52.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:52 smithi050 ceph-mon[30378]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:53.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:52 smithi142 ceph-mon[35329]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:54 smithi082 ceph-mon[35046]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:22:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:54 smithi050 ceph-mon[30378]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:22:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:54 smithi142 ceph-mon[35329]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:22:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:55.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:55 smithi082 ceph-mon[35046]: Deploying daemon osd.10 on smithi082 2024-09-12T03:22:55.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:55 smithi050 ceph-mon[30378]: Deploying daemon osd.10 on smithi082 2024-09-12T03:22:56.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:55 smithi142 ceph-mon[35329]: Deploying daemon osd.10 on smithi082 2024-09-12T03:22:56.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:56 smithi082 ceph-mon[35046]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:56.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:56 smithi050 ceph-mon[30378]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:56.945 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:56 smithi142 ceph-mon[35329]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:22:58 smithi082 ceph-mon[35046]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:58.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:22:58 smithi050 ceph-mon[30378]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:22:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:22:58 smithi142 ceph-mon[35329]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:00 smithi082 ceph-mon[35046]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:00.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:00 smithi050 ceph-mon[30378]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:00.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:00.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:00.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:00 smithi142 ceph-mon[35329]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:01.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:01.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:02 smithi050 ceph-mon[30378]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:03.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:02 smithi142 ceph-mon[35329]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:03.061 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:02 smithi082 ceph-mon[35046]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:03.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.390+0000 7fea6d7fa640 1 -- 172.21.15.82:0/3883332006 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fea70062630 con 0x7fea5c0a2b80 2024-09-12T03:23:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 10 on host 'smithi082' 2024-09-12T03:23:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fea5c0a2b80 msgr2=0x7fea5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fea5c0a2b80 0x7fea5c0a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea60005ab0 tx=0x7fea60000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 msgr2=0x7fea70109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 0x7fea70109fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fea6800eaa0 tx=0x7fea6800ef60 comp rx=0 tx=0).stop 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 shutdown_connections 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fea5c0a2b80 0x7fea5c0a5040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea70102690 0x7fea70109fe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea70101cd0 0x7fea70109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 --2- 172.21.15.82:0/3883332006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea70100ad0 0x7fea7010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 >> 172.21.15.82:0/3883332006 conn(0x7fea700fc280 msgr2=0x7fea701050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 shutdown_connections 2024-09-12T03:23:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-09-12T03:23:03.392+0000 7fea75e49640 1 -- 172.21.15.82:0/3883332006 wait complete. 2024-09-12T03:23:04.151 DEBUG:teuthology.orchestra.run.smithi082:osd.10> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T03:23:04.154 INFO:tasks.cephadm:Deploying osd.11 on smithi142 with /dev/vg_nvme/lv_1... 2024-09-12T03:23:04.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:23:04.411 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:23:04.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:04 smithi142 ceph-mon[35329]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:04.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:04.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:04 smithi050 ceph-mon[30378]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:04 smithi082 ceph-mon[35046]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:04 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:04 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.006 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:05 smithi142 ceph-mon[35329]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:06.006 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:05 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.006 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:05 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:05 smithi082 ceph-mon[35046]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:05 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:05 smithi050 ceph-mon[30378]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:06.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:05 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:05 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:06.739 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:23:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:23:06.575+0000 7f1e2fa76640 -1 osd.10 0 log_to_monitors true 2024-09-12T03:23:07.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:06 smithi082 ceph-mon[35046]: from='osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:23:07.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:06 smithi082 ceph-mon[35046]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:23:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:06 smithi050 ceph-mon[30378]: from='osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:23:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:06 smithi050 ceph-mon[30378]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:23:07.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:06 smithi142 ceph-mon[35329]: from='osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:23:07.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:06 smithi142 ceph-mon[35329]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:23:07.319 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T03:23:07.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-09-12T03:23:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:23:07.952 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:07 smithi082 ceph-mon[35046]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:07.952 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:07 smithi082 ceph-mon[35046]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:23:07.953 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:07 smithi082 ceph-mon[35046]: from='osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:23:07.953 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:07 smithi082 ceph-mon[35046]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:23:07.953 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:07 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:07.953 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:07 smithi082 ceph-mon[35046]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:23:08.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:07 smithi142 ceph-mon[35329]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:08.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:07 smithi142 ceph-mon[35329]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:23:08.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:07 smithi142 ceph-mon[35329]: from='osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:23:08.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:07 smithi142 ceph-mon[35329]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:23:08.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:07 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:08.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:07 smithi142 ceph-mon[35329]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:23:08.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 -- 172.21.15.142:0/2962764803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60101c00 msgr2=0x7fcf60102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2962764803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60101c00 0x7fcf60102000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009900 tx=0x7fcf5c02a410 comp rx=0 tx=0).stop 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 -- 172.21.15.142:0/2962764803 shutdown_connections 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2962764803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf601037c0 0x7fcf6010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2962764803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60102e00 0x7fcf60103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2962764803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60101c00 0x7fcf60102000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 -- 172.21.15.142:0/2962764803 >> 172.21.15.142:0/2962764803 conn(0x7fcf600fd390 msgr2=0x7fcf600ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 -- 172.21.15.142:0/2962764803 shutdown_connections 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 -- 172.21.15.142:0/2962764803 wait complete. 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.111+0000 7fcf6835b640 1 Processor -- start 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 -- start start 2024-09-12T03:23:08.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf60101c00 0x7fcf60075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 0x7fcf600745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf601037c0 0x7fcf60074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d900 con 0x7fcf601037c0 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010d780 con 0x7fcf60101c00 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf6835b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010da80 con 0x7fcf60102e00 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 0x7fcf600745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 0x7fcf600745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39820/0 (socket says 172.21.15.142:39820) 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 -- 172.21.15.142:0/2726621111 learned_addr learned my addr 172.21.15.142:0/2726621111 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf660f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf60101c00 0x7fcf60075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 -- 172.21.15.142:0/2726621111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf60101c00 msgr2=0x7fcf60075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf60101c00 0x7fcf60075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 -- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf601037c0 msgr2=0x7fcf60074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf601037c0 0x7fcf60074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 -- 172.21.15.142:0/2726621111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf600753e0 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf658f6640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 0x7fcf600745e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b4a0 tx=0x7fcf5000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.112+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50014070 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.113+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a20e0 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.113+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50010af0 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.113+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50017d10 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.113+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a26c0 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.113+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf60109050 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.115+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf50017070 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.115+0000 7fcf577fe640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.115+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7fcf500ce010 con 0x7fcf60102e00 2024-09-12T03:23:08.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.116+0000 7fcf660f7640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.116+0000 7fcf660f7640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c005ab0 tx=0x7fcf5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.119+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf5009b6e0 con 0x7fcf60102e00 2024-09-12T03:23:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:07 smithi050 ceph-mon[30378]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:07 smithi050 ceph-mon[30378]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:23:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:07 smithi050 ceph-mon[30378]: from='osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:23:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:07 smithi050 ceph-mon[30378]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:23:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:07 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:07 smithi050 ceph-mon[30378]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-09-12T03:23:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:08.300+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcf60062630 con 0x7fcf4c0a2b80 2024-09-12T03:23:08.832 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:23:08.567+0000 7f1e2b9f9640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T03:23:08.832 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:23:08.580+0000 7f1e23819640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:23:08.982 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:23:08.982 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T03:23:08.982 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:08.982 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi082 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi082 to 4272M 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:08.983 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:08.984 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:08 smithi142 ceph-mon[35329]: from='osd.10 ' entity='osd.10' 2024-09-12T03:23:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:23:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi082 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi082 to 4272M 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:09.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:09.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:09.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:09.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T03:23:09.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:09.107 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:08 smithi082 ceph-mon[35046]: from='osd.10 ' entity='osd.10' 2024-09-12T03:23:09.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi082 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:09.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi082 to 4272M 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:09.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T03:23:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:09.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:08 smithi050 ceph-mon[30378]: from='osd.10 ' entity='osd.10' 2024-09-12T03:23:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:23:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:23:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] boot 2024-09-12T03:23:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:23:10.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] boot 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:23:10.164 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:10.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:23:10.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:23:10.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:23:10.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:10.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: osd.10 [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] boot 2024-09-12T03:23:10.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:23:10.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:23:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T03:23:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/138263088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]: dispatch 2024-09-12T03:23:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]: dispatch 2024-09-12T03:23:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]': finished 2024-09-12T03:23:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:23:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:11.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:11 smithi142 ceph-mon[35329]: from='client.? 172.21.15.142:0/276090409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/138263088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]: dispatch 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]: dispatch 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]': finished 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:11.811 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:11 smithi082 ceph-mon[35046]: from='client.? 172.21.15.142:0/276090409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:11.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T03:23:11.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/138263088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]: dispatch 2024-09-12T03:23:11.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]: dispatch 2024-09-12T03:23:11.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d5befe3-b567-4cd2-b31a-a8d06d402568"}]': finished 2024-09-12T03:23:11.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:23:11.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:11.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:11 smithi050 ceph-mon[30378]: from='client.? 172.21.15.142:0/276090409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:12 smithi142 ceph-mon[35329]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:12 smithi082 ceph-mon[35046]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:12 smithi050 ceph-mon[30378]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:13.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:13 smithi142 ceph-mon[35329]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:23:13.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:13 smithi082 ceph-mon[35046]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:23:13.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:13 smithi050 ceph-mon[30378]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:23:13.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:14 smithi142 ceph-mon[35329]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:14 smithi142 ceph-mon[35329]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:23:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:14 smithi082 ceph-mon[35046]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:14 smithi082 ceph-mon[35046]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:23:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:14 smithi050 ceph-mon[30378]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:14 smithi050 ceph-mon[30378]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:23:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:16 smithi142 ceph-mon[35329]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-12T03:23:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:16 smithi082 ceph-mon[35046]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-12T03:23:16.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:16 smithi050 ceph-mon[30378]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-12T03:23:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:18 smithi050 ceph-mon[30378]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:23:18.725 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:18 smithi142 ceph-mon[35329]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:23:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:18 smithi082 ceph-mon[35046]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:23:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:20 smithi142 ceph-mon[35329]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-12T03:23:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:23:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:20 smithi082 ceph-mon[35046]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-12T03:23:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:23:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:20 smithi050 ceph-mon[30378]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-12T03:23:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:23:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:21.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:21 smithi142 ceph-mon[35329]: Deploying daemon osd.11 on smithi142 2024-09-12T03:23:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:21 smithi082 ceph-mon[35046]: Deploying daemon osd.11 on smithi142 2024-09-12T03:23:21.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:21 smithi050 ceph-mon[30378]: Deploying daemon osd.11 on smithi142 2024-09-12T03:23:22.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:22 smithi142 ceph-mon[35329]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:23:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:22 smithi082 ceph-mon[35046]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:23:22.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:22 smithi050 ceph-mon[30378]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T03:23:24.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:24 smithi142 ceph-mon[35329]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:23:24.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:24 smithi082 ceph-mon[35046]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:23:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:24 smithi050 ceph-mon[30378]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:23:26.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:26 smithi142 ceph-mon[35329]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:26.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:26 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:26.621 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:26 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:26.621 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:26 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:26 smithi082 ceph-mon[35046]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:26 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:26 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:26 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:26.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:26 smithi050 ceph-mon[30378]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:26.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:26 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:26.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:26 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:26.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:26 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:28.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:28 smithi142 ceph-mon[35329]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:28.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:28 smithi082 ceph-mon[35046]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:28.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:28 smithi050 ceph-mon[30378]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:29.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.631+0000 7fcf577fe640 1 -- 172.21.15.142:0/2726621111 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcf60062630 con 0x7fcf4c0a2b80 2024-09-12T03:23:29.631 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 11 on host 'smithi142' 2024-09-12T03:23:29.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcf4c0a2b80 msgr2=0x7fcf4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c005ab0 tx=0x7fcf5c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 msgr2=0x7fcf600745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 0x7fcf600745e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b4a0 tx=0x7fcf5000b960 comp rx=0 tx=0).stop 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 shutdown_connections 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf601037c0 0x7fcf60074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60102e00 0x7fcf600745e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 --2- 172.21.15.142:0/2726621111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf60101c00 0x7fcf60075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 >> 172.21.15.142:0/2726621111 conn(0x7fcf600fd390 msgr2=0x7fcf60106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 shutdown_connections 2024-09-12T03:23:29.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:29.633+0000 7fcf6835b640 1 -- 172.21.15.142:0/2726621111 wait complete. 2024-09-12T03:23:30.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:30.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:30.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:30.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:30.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.352 DEBUG:teuthology.orchestra.run.smithi142:osd.11> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T03:23:30.354 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T03:23:30.354 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:23:30.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:30.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:30.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:30.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:30.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:30.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:30.742 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:31.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 -- 172.21.15.50:0/4002200060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014104f60 msgr2=0x7f60141053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 --2- 172.21.15.50:0/4002200060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014104f60 0x7f60141053e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6004009860 tx=0x7f600402a340 comp rx=0 tx=0).stop 2024-09-12T03:23:31.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 -- 172.21.15.50:0/4002200060 shutdown_connections 2024-09-12T03:23:31.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 --2- 172.21.15.50:0/4002200060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 0x7f601410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 --2- 172.21.15.50:0/4002200060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014104f60 0x7f60141053e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 --2- 172.21.15.50:0/4002200060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6014103e00 0x7f6014104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 -- 172.21.15.50:0/4002200060 >> 172.21.15.50:0/4002200060 conn(0x7f60140ff5b0 msgr2=0x7f60141019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 -- 172.21.15.50:0/4002200060 shutdown_connections 2024-09-12T03:23:31.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.408+0000 7f601c800640 1 -- 172.21.15.50:0/4002200060 wait complete. 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 Processor -- start 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 -- start start 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 0x7f601419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6014104f60 0x7f601419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 0x7f60141a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f601410fa90 con 0x7f6014105920 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f601410f910 con 0x7f6014103e00 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f601c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f601410fc10 con 0x7f6014104f60 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.409+0000 7f6019e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 0x7f601419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 0x7f601419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:40842/0 (socket says 172.21.15.50:40842) 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 -- 172.21.15.50:0/1627830078 learned_addr learned my addr 172.21.15.50:0/1627830078 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f601a800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 0x7f60141a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019400640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6014104f60 0x7f601419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 -- 172.21.15.50:0/1627830078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6014104f60 msgr2=0x7f601419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6014104f60 0x7f601419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 -- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 msgr2=0x7f60141a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 0x7f60141a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 -- 172.21.15.50:0/1627830078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60141a1d10 con 0x7f6014103e00 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f601a800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 0x7f60141a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f6019e00640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 0x7f601419a0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f600800e9f0 tx=0x7f600800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800cd80 con 0x7f6014103e00 2024-09-12T03:23:31.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.410+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008010910 con 0x7f6014103e00 2024-09-12T03:23:31.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.411+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008018990 con 0x7f6014103e00 2024-09-12T03:23:31.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.411+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60141a1fa0 con 0x7f6014103e00 2024-09-12T03:23:31.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.411+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141a24a0 con 0x7f6014103e00 2024-09-12T03:23:31.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.412+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601410b240 con 0x7f6014103e00 2024-09-12T03:23:31.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.414+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6008010ac0 con 0x7f6014103e00 2024-09-12T03:23:31.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.415+0000 7f600ea00640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5ffc0a2c30 0x7f5ffc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.415+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7f60080cf370 con 0x7f6014103e00 2024-09-12T03:23:31.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.415+0000 7f6019400640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5ffc0a2c30 0x7f5ffc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.416+0000 7f6019400640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5ffc0a2c30 0x7f5ffc0a50f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6004030410 tx=0x7f6004030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.417+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f600809c810 con 0x7f6014103e00 2024-09-12T03:23:31.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.603+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f601402d7c0 con 0x7f6014103e00 2024-09-12T03:23:31.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.604+0000 7f600ea00640 1 -- 172.21.15.50:0/1627830078 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f600809c110 con 0x7f6014103e00 2024-09-12T03:23:31.604 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5ffc0a2c30 msgr2=0x7f5ffc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5ffc0a2c30 0x7f5ffc0a50f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6004030410 tx=0x7f6004030550 comp rx=0 tx=0).stop 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 msgr2=0x7f601419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 0x7f601419a0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f600800e9f0 tx=0x7f600800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 shutdown_connections 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5ffc0a2c30 0x7f5ffc0a50f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6014105920 0x7f60141a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6014104f60 0x7f601419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 --2- 172.21.15.50:0/1627830078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6014103e00 0x7f601419a0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 >> 172.21.15.50:0/1627830078 conn(0x7f60140ff5b0 msgr2=0x7f6014108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 shutdown_connections 2024-09-12T03:23:31.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:31.606+0000 7f601c800640 1 -- 172.21.15.50:0/1627830078 wait complete. 2024-09-12T03:23:32.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:32 smithi142 ceph-mon[35329]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:32.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:32 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1627830078' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:23:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:23:32.197+0000 7f8caf216640 -1 osd.11 0 log_to_monitors true 2024-09-12T03:23:32.596 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:32 smithi050 ceph-mon[30378]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:32.596 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:32 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1627830078' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:32.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:32 smithi082 ceph-mon[35046]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:32.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:32 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1627830078' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:32.634 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726111389,"num_in_osds":12,"osd_in_since":1726111390,"num_remapped_pgs":0} 2024-09-12T03:23:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:33 smithi142 ceph-mon[35329]: from='osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:23:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:33 smithi142 ceph-mon[35329]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:23:33.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:33 smithi082 ceph-mon[35046]: from='osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:23:33.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:33 smithi082 ceph-mon[35046]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:23:33.635 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:23:33.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:33 smithi050 ceph-mon[30378]: from='osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:23:33.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:33 smithi050 ceph-mon[30378]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:23:34.008 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: Detected new or changed devices on smithi142 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-12T03:23:34.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:34.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:34.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:23:34.219+0000 7f8cab199640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T03:23:34.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:23:34.231+0000 7f8ca6fc1640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: Detected new or changed devices on smithi142 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:34.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:23:34.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:23:34.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:23:34.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: Detected new or changed devices on smithi142 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:34.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 -- 172.21.15.50:0/93298762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 msgr2=0x7fceb8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 --2- 172.21.15.50:0/93298762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb8102000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009900 tx=0x7fcea802a410 comp rx=0 tx=0).stop 2024-09-12T03:23:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 -- 172.21.15.50:0/93298762 shutdown_connections 2024-09-12T03:23:34.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 --2- 172.21.15.50:0/93298762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb81037c0 0x7fceb810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 --2- 172.21.15.50:0/93298762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 0x7fceb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 --2- 172.21.15.50:0/93298762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb8102000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 -- 172.21.15.50:0/93298762 >> 172.21.15.50:0/93298762 conn(0x7fceb80fd390 msgr2=0x7fceb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 -- 172.21.15.50:0/93298762 shutdown_connections 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.659+0000 7fcebcc00640 1 -- 172.21.15.50:0/93298762 wait complete. 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 Processor -- start 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 -- start start 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 0x7fceb819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb81037c0 0x7fceb81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceb810d8e0 con 0x7fceb8101c00 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceb810d760 con 0x7fceb8102e00 2024-09-12T03:23:34.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fcebcc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fceb810da60 con 0x7fceb81037c0 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.660+0000 7fceb5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 0x7fceb819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52668/0 (socket says 172.21.15.50:52668) 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5e00640 1 -- 172.21.15.50:0/641106629 learned_addr learned my addr 172.21.15.50:0/641106629 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb6800640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb81037c0 0x7fceb81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 -- 172.21.15.50:0/641106629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb81037c0 msgr2=0x7fceb81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb81037c0 0x7fceb81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 -- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 msgr2=0x7fceb819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 -- 172.21.15.50:0/641106629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb81a1d80 con 0x7fceb8102e00 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.661+0000 7fceb5400640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 0x7fceb819a690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcea400e9f0 tx=0x7fcea400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.662+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400ccc0 con 0x7fceb8102e00 2024-09-12T03:23:34.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.662+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea4010910 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.662+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4018960 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.662+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb81a2070 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.662+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb81a2560 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.663+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb81a27c0 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.665+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcea4010ac0 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.666+0000 7fceaea00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2bb0 0x7fce9c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.666+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6847+0+0 (secure 0 0 0) 0x7fcea40cf240 con 0x7fceb8102e00 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.666+0000 7fceb5e00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2bb0 0x7fce9c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.667+0000 7fceb5e00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2bb0 0x7fce9c0a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.668+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcea409c070 con 0x7fceb8102e00 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.855+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fceb81a2a30 con 0x7fceb8102e00 2024-09-12T03:23:34.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.856+0000 7fceaea00640 1 -- 172.21.15.50:0/641106629 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fcea40a0370 con 0x7fceb8102e00 2024-09-12T03:23:34.856 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:34.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2bb0 msgr2=0x7fce9c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2bb0 0x7fce9c0a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea80315b0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 msgr2=0x7fceb819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 0x7fceb819a690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcea400e9f0 tx=0x7fcea400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 shutdown_connections 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2bb0 0x7fce9c0a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb81037c0 0x7fceb81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb8102e00 0x7fceb819a690 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 --2- 172.21.15.50:0/641106629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb8101c00 0x7fceb819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.858+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 >> 172.21.15.50:0/641106629 conn(0x7fceb80fd390 msgr2=0x7fceb8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.859+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 shutdown_connections 2024-09-12T03:23:34.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:34.859+0000 7fcebcc00640 1 -- 172.21.15.50:0/641106629 wait complete. 2024-09-12T03:23:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: purged_snaps scrub starts 2024-09-12T03:23:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: purged_snaps scrub ok 2024-09-12T03:23:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:23:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T03:23:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:35.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:35.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:35 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/641106629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:35.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: purged_snaps scrub starts 2024-09-12T03:23:35.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: purged_snaps scrub ok 2024-09-12T03:23:35.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:23:35.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T03:23:35.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:35.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:35.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:35 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/641106629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:35.553 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1726111389,"num_in_osds":12,"osd_in_since":1726111390,"num_remapped_pgs":0} 2024-09-12T03:23:35.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: purged_snaps scrub starts 2024-09-12T03:23:35.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: purged_snaps scrub ok 2024-09-12T03:23:35.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-12T03:23:35.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T03:23:35.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:35.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:35.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:35 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/641106629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:36.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:36 smithi142 ceph-mon[35329]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:36.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:36 smithi142 ceph-mon[35329]: osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] boot 2024-09-12T03:23:36.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:36 smithi142 ceph-mon[35329]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:23:36.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:36 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:36.554 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:23:36.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:36 smithi082 ceph-mon[35046]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:36.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:36 smithi082 ceph-mon[35046]: osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] boot 2024-09-12T03:23:36.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:36 smithi082 ceph-mon[35046]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:23:36.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:36 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:36.608 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:36 smithi050 ceph-mon[30378]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:23:36.608 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:36 smithi050 ceph-mon[30378]: osd.11 [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] boot 2024-09-12T03:23:36.608 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:36 smithi050 ceph-mon[30378]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:23:36.608 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:36 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:23:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:37.335 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:37 smithi050 ceph-mon[30378]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:23:37.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 -- 172.21.15.50:0/511160123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 msgr2=0x7ff320102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 --2- 172.21.15.50:0/511160123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320102000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff310009900 tx=0x7ff31002a410 comp rx=0 tx=0).stop 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 -- 172.21.15.50:0/511160123 shutdown_connections 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 --2- 172.21.15.50:0/511160123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 0x7ff32010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 --2- 172.21.15.50:0/511160123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320102e00 0x7ff320103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 --2- 172.21.15.50:0/511160123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320102000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.448+0000 7ff324a00640 1 -- 172.21.15.50:0/511160123 >> 172.21.15.50:0/511160123 conn(0x7ff3200fd390 msgr2=0x7ff3200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.449+0000 7ff324a00640 1 -- 172.21.15.50:0/511160123 shutdown_connections 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.449+0000 7ff324a00640 1 -- 172.21.15.50:0/511160123 wait complete. 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 Processor -- start 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 -- start start 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320102e00 0x7ff320198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 0x7ff32019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff32010d8e0 con 0x7ff320101c00 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff32010d760 con 0x7ff320102e00 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.450+0000 7ff324a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff32010da60 con 0x7ff3201037c0 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52700/0 (socket says 172.21.15.50:52700) 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 -- 172.21.15.50:0/3981849801 learned_addr learned my addr 172.21.15.50:0/3981849801 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31e800640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 0x7ff32019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 -- 172.21.15.50:0/3981849801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 msgr2=0x7ff32019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 0x7ff32019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 -- 172.21.15.50:0/3981849801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320102e00 msgr2=0x7ff320198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320102e00 0x7ff320198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 -- 172.21.15.50:0/3981849801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32019fb60 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31e800640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 0x7ff32019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.451+0000 7ff31de00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320197f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff310031440 tx=0x7ff310032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.452+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31003f030 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.452+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff310003ce0 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.452+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310035070 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.452+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32019fdf0 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.452+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3201a0250 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.453+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff320109000 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.455+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff310042030 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.457+0000 7ff316a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff3040a2c30 0x7ff3040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.457+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff3100ef0d0 con 0x7ff320101c00 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.457+0000 7ff317e00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff3040a2c30 0x7ff3040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.458+0000 7ff317e00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff3040a2c30 0x7ff3040a50f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3080058c0 tx=0x7ff308005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.461+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3100bc430 con 0x7ff320101c00 2024-09-12T03:23:37.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:37 smithi142 ceph-mon[35329]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:23:37.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:37 smithi082 ceph-mon[35046]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:23:37.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.647+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff320062ef0 con 0x7ff320101c00 2024-09-12T03:23:37.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.648+0000 7ff316a00640 1 -- 172.21.15.50:0/3981849801 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7ff3100bbd30 con 0x7ff320101c00 2024-09-12T03:23:37.647 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff3040a2c30 msgr2=0x7ff3040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff3040a2c30 0x7ff3040a50f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3080058c0 tx=0x7ff308005850 comp rx=0 tx=0).stop 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 msgr2=0x7ff320197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320197f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff310031440 tx=0x7ff310032ce0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 shutdown_connections 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff3040a2c30 0x7ff3040a50f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3201037c0 0x7ff32019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320102e00 0x7ff320198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 --2- 172.21.15.50:0/3981849801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff320101c00 0x7ff320197f30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 >> 172.21.15.50:0/3981849801 conn(0x7ff3200fd390 msgr2=0x7ff3201061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 shutdown_connections 2024-09-12T03:23:37.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:37.650+0000 7ff324a00640 1 -- 172.21.15.50:0/3981849801 wait complete. 2024-09-12T03:23:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:38 smithi050 ceph-mon[30378]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:38.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:38 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3981849801' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:38.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:38 smithi142 ceph-mon[35329]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:38.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:38 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3981849801' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:38.570 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726111415,"num_in_osds":12,"osd_in_since":1726111390,"num_remapped_pgs":0} 2024-09-12T03:23:38.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:23:38.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:38 smithi082 ceph-mon[35046]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:38.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:38 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3981849801' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:23:38.946 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:39.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 -- 172.21.15.50:0/3131280377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb78103ff0 msgr2=0x7fcb781043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3131280377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb78103ff0 0x7fcb781043d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009900 tx=0x7fcb6802a410 comp rx=0 tx=0).stop 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 -- 172.21.15.50:0/3131280377 shutdown_connections 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3131280377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb78109120 0x7fcb78110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3131280377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb781049a0 0x7fcb781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3131280377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb78103ff0 0x7fcb781043d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 -- 172.21.15.50:0/3131280377 >> 172.21.15.50:0/3131280377 conn(0x7fcb780ffa90 msgr2=0x7fcb78101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 -- 172.21.15.50:0/3131280377 shutdown_connections 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.465+0000 7fcb7d600640 1 -- 172.21.15.50:0/3131280377 wait complete. 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.466+0000 7fcb7d600640 1 Processor -- start 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.466+0000 7fcb7d600640 1 -- start start 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.466+0000 7fcb7d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb78103ff0 0x7fcb7819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.466+0000 7fcb7d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 0x7fcb7819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.466+0000 7fcb7d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb78109120 0x7fcb7819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb7d600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78114170 con 0x7fcb781049a0 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb7d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78113ff0 con 0x7fcb78109120 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb7d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb781142f0 con 0x7fcb78103ff0 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 0x7fcb7819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 0x7fcb7819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37190/0 (socket says 172.21.15.50:37190) 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 -- 172.21.15.50:0/3987518205 learned_addr learned my addr 172.21.15.50:0/3987518205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb77400640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb78109120 0x7fcb7819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb76a00640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb78103ff0 0x7fcb7819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 -- 172.21.15.50:0/3987518205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb78103ff0 msgr2=0x7fcb7819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb78103ff0 0x7fcb7819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 -- 172.21.15.50:0/3987518205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb78109120 msgr2=0x7fcb7819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb78109120 0x7fcb7819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb6e000640 1 -- 172.21.15.50:0/3987518205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7819f620 con 0x7fcb781049a0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.467+0000 7fcb77400640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb78109120 0x7fcb7819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.468+0000 7fcb6e000640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 0x7fcb7819abb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.468+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000cd80 con 0x7fcb781049a0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.468+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb60010910 con 0x7fcb781049a0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.468+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018990 con 0x7fcb781049a0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.468+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7819f910 con 0x7fcb781049a0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.468+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb781a71f0 con 0x7fcb781049a0 2024-09-12T03:23:39.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.469+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb781043d0 con 0x7fcb781049a0 2024-09-12T03:23:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.472+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcb60010ac0 con 0x7fcb781049a0 2024-09-12T03:23:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.473+0000 7fcb6fe00640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcb5c0a2c00 0x7fcb5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.473+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fcb600cefe0 con 0x7fcb781049a0 2024-09-12T03:23:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.473+0000 7fcb76a00640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcb5c0a2c00 0x7fcb5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:39.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.474+0000 7fcb76a00640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcb5c0a2c00 0x7fcb5c0a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000c00 tx=0x7fcb68005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:39.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.477+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb6009bcf0 con 0x7fcb781049a0 2024-09-12T03:23:39.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.661+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb7810bd70 con 0x7fcb781049a0 2024-09-12T03:23:39.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.662+0000 7fcb6fe00640 1 -- 172.21.15.50:0/3987518205 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18410 (secure 0 0 0) 0x7fcb6009fff0 con 0x7fcb781049a0 2024-09-12T03:23:39.663 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:39.663 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":78,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:23:36.226152+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcb5c0a2c00 msgr2=0x7fcb5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcb5c0a2c00 0x7fcb5c0a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000c00 tx=0x7fcb68005b50 comp rx=0 tx=0).stop 2024-09-12T03:23:39.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 msgr2=0x7fcb7819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 0x7fcb7819abb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 shutdown_connections 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcb5c0a2c00 0x7fcb5c0a50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb78109120 0x7fcb7819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb781049a0 0x7fcb7819abb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 --2- 172.21.15.50:0/3987518205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb78103ff0 0x7fcb7819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.665+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 >> 172.21.15.50:0/3987518205 conn(0x7fcb780ffa90 msgr2=0x7fcb78100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.666+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 shutdown_connections 2024-09-12T03:23:39.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:39.666+0000 7fcb7d600640 1 -- 172.21.15.50:0/3987518205 wait complete. 2024-09-12T03:23:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:40 smithi050 ceph-mon[30378]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:40 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3987518205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:23:40.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:40 smithi142 ceph-mon[35329]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:40.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:40 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3987518205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:23:40.562 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:19:33.283337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:23:40.563 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T03:23:40.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:40 smithi082 ceph-mon[35046]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:40.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:40 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3987518205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:23:40.932 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:41.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 -- 172.21.15.50:0/415465589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1040103ff0 msgr2=0x7f10401043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 --2- 172.21.15.50:0/415465589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1040103ff0 0x7f10401043d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1030009900 tx=0x7f103002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 -- 172.21.15.50:0/415465589 shutdown_connections 2024-09-12T03:23:41.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 --2- 172.21.15.50:0/415465589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1040109120 0x7f1040110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 --2- 172.21.15.50:0/415465589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401049a0 0x7f10401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 --2- 172.21.15.50:0/415465589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1040103ff0 0x7f10401043d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 -- 172.21.15.50:0/415465589 >> 172.21.15.50:0/415465589 conn(0x7f10400ffa90 msgr2=0x7f1040101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:41.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 -- 172.21.15.50:0/415465589 shutdown_connections 2024-09-12T03:23:41.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.461+0000 7f1046e00640 1 -- 172.21.15.50:0/415465589 wait complete. 2024-09-12T03:23:41.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.462+0000 7f1046e00640 1 Processor -- start 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.462+0000 7f1046e00640 1 -- start start 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1046e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1040103ff0 0x7f104019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1046e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 0x7f104019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1046e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1040109120 0x7f104019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1046e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1040114190 con 0x7f10401049a0 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1046e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1040114010 con 0x7f1040109120 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1046e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1040114310 con 0x7f1040103ff0 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1037e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 0x7f104019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1044e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1040109120 0x7f104019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1037e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 0x7f104019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37210/0 (socket says 172.21.15.50:37210) 2024-09-12T03:23:41.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f1037e00640 1 -- 172.21.15.50:0/938120409 learned_addr learned my addr 172.21.15.50:0/938120409 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.463+0000 7f103fe00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1040103ff0 0x7f104019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1037e00640 1 -- 172.21.15.50:0/938120409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1040103ff0 msgr2=0x7f104019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1037e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1040103ff0 0x7f104019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1037e00640 1 -- 172.21.15.50:0/938120409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1040109120 msgr2=0x7f104019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1037e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1040109120 0x7f104019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1037e00640 1 -- 172.21.15.50:0/938120409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104019f5c0 con 0x7f10401049a0 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1037e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 0x7f104019ab50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f102800d770 tx=0x7f102800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:41.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028017070 con 0x7f10401049a0 2024-09-12T03:23:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1028005500 con 0x7f10401049a0 2024-09-12T03:23:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028004e00 con 0x7f10401049a0 2024-09-12T03:23:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104019f8b0 con 0x7f10401049a0 2024-09-12T03:23:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.464+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10401a7190 con 0x7f10401049a0 2024-09-12T03:23:41.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.466+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10401043d0 con 0x7f10401049a0 2024-09-12T03:23:41.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.469+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1028004fa0 con 0x7f10401049a0 2024-09-12T03:23:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.470+0000 7f103d600640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f10240a2c00 0x7f10240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.470+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f10280ce9d0 con 0x7f10401049a0 2024-09-12T03:23:41.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.471+0000 7f103fe00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f10240a2c00 0x7f10240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:41.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.471+0000 7f103fe00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f10240a2c00 0x7f10240a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1030000c00 tx=0x7f1030005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:41.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.473+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f102809b760 con 0x7f10401049a0 2024-09-12T03:23:41.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.658+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10401a73f0 con 0x7f10401049a0 2024-09-12T03:23:41.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.658+0000 7f103d600640 1 -- 172.21.15.50:0/938120409 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f102809fa60 con 0x7f10401049a0 2024-09-12T03:23:41.659 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f10240a2c00 msgr2=0x7f10240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f10240a2c00 0x7f10240a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1030000c00 tx=0x7f1030005b50 comp rx=0 tx=0).stop 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 msgr2=0x7f104019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 0x7f104019ab50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f102800d770 tx=0x7f102800dc30 comp rx=0 tx=0).stop 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 shutdown_connections 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f10240a2c00 0x7f10240a50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1040109120 0x7f104019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10401049a0 0x7f104019ab50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 --2- 172.21.15.50:0/938120409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1040103ff0 0x7f104019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 >> 172.21.15.50:0/938120409 conn(0x7f10400ffa90 msgr2=0x7f10401005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:41.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 shutdown_connections 2024-09-12T03:23:41.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:41.661+0000 7f1046e00640 1 -- 172.21.15.50:0/938120409 wait complete. 2024-09-12T03:23:42.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:42 smithi050 ceph-mon[30378]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:42.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:42 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/938120409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:23:42.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:42 smithi142 ceph-mon[35329]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:42.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:42 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/938120409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.a on smithi050 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.d on smithi050 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.g on smithi050 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.j on smithi050 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.b on smithi082 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.e on smithi082 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.h on smithi082 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.k on smithi082 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.c on smithi142 2024-09-12T03:23:42.584 INFO:tasks.cephadm:Adding mds.f on smithi142 2024-09-12T03:23:42.585 INFO:tasks.cephadm:Adding mds.i on smithi142 2024-09-12T03:23:42.585 INFO:tasks.cephadm:Adding mds.l on smithi142 2024-09-12T03:23:42.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l' 2024-09-12T03:23:42.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:42 smithi082 ceph-mon[35046]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:42.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:42 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/938120409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:23:42.846 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 -- 172.21.15.142:0/1137670919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b8101a30 msgr2=0x7f44b8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1137670919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b8101a30 0x7f44b8101e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009900 tx=0x7f44a802a410 comp rx=0 tx=0).stop 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 -- 172.21.15.142:0/1137670919 shutdown_connections 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1137670919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44b81035f0 0x7f44b8109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1137670919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b8102c30 0x7f44b81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1137670919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b8101a30 0x7f44b8101e30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 -- 172.21.15.142:0/1137670919 >> 172.21.15.142:0/1137670919 conn(0x7f44b80fd1c0 msgr2=0x7f44b80ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 -- 172.21.15.142:0/1137670919 shutdown_connections 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.350+0000 7f44bd79d640 1 -- 172.21.15.142:0/1137670919 wait complete. 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 Processor -- start 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 -- start start 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b8101a30 0x7f44b8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44b8102c30 0x7f44b8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 0x7f44b819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810d710 con 0x7f44b8102c30 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810d590 con 0x7f44b8101a30 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bd79d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810d890 con 0x7f44b81035f0 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.351+0000 7f44bcf9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 0x7f44b819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 0x7f44b819f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43098/0 (socket says 172.21.15.142:43098) 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 -- 172.21.15.142:0/1652021852 learned_addr learned my addr 172.21.15.142:0/1652021852 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T03:23:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44b7fff640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b8101a30 0x7f44b8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44b77fe640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44b8102c30 0x7f44b8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 -- 172.21.15.142:0/1652021852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b8101a30 msgr2=0x7f44b8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b8101a30 0x7f44b8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 -- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44b8102c30 msgr2=0x7f44b8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44b8102c30 0x7f44b8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 -- 172.21.15.142:0/1652021852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b819fa20 con 0x7f44b81035f0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bcf9c640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 0x7f44b819f3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f44b000cad0 tx=0x7f44b000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000ad60 con 0x7f44b81035f0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b819fd10 con 0x7f44b81035f0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b0005020 con 0x7f44b81035f0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000f6b0 con 0x7f44b81035f0 2024-09-12T03:23:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.352+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b81a0270 con 0x7f44b81035f0 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.353+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b8108ef0 con 0x7f44b81035f0 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.355+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44b0005560 con 0x7f44b81035f0 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.355+0000 7f44b57fa640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a00a2bb0 0x7f44a00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.355+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f44b00ceeb0 con 0x7f44b81035f0 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.356+0000 7f44b7fff640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a00a2bb0 0x7f44a00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.356+0000 7f44b7fff640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a00a2bb0 0x7f44a00a5070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005ab0 tx=0x7f44a8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.359+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44b00fe1b0 con 0x7f44b81035f0 2024-09-12T03:23:43.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.543+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}) -- 0x7f44b8062730 con 0x7f44a00a2bb0 2024-09-12T03:23:43.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.551+0000 7f44b57fa640 1 -- 172.21.15.142:0/1652021852 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f44b8062730 con 0x7f44a00a2bb0 2024-09-12T03:23:43.551 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mds.all update... 2024-09-12T03:23:43.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a00a2bb0 msgr2=0x7f44a00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a00a2bb0 0x7f44a00a5070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005ab0 tx=0x7f44a8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 msgr2=0x7f44b819f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 0x7f44b819f3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f44b000cad0 tx=0x7f44b000ced0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 shutdown_connections 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a00a2bb0 0x7f44a00a5070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44b81035f0 0x7f44b819f3b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44b8102c30 0x7f44b8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 --2- 172.21.15.142:0/1652021852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b8101a30 0x7f44b8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 >> 172.21.15.142:0/1652021852 conn(0x7f44b80fd1c0 msgr2=0x7f44b81060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 shutdown_connections 2024-09-12T03:23:43.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T03:23:43.553+0000 7f44bd79d640 1 -- 172.21.15.142:0/1652021852 wait complete. 2024-09-12T03:23:44.326 DEBUG:teuthology.orchestra.run.smithi050:mds.a> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T03:23:44.331 DEBUG:teuthology.orchestra.run.smithi050:mds.d> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T03:23:44.375 DEBUG:teuthology.orchestra.run.smithi050:mds.g> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T03:23:44.421 DEBUG:teuthology.orchestra.run.smithi050:mds.j> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T03:23:44.466 DEBUG:teuthology.orchestra.run.smithi082:mds.b> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T03:23:44.469 DEBUG:teuthology.orchestra.run.smithi082:mds.e> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T03:23:44.517 DEBUG:teuthology.orchestra.run.smithi082:mds.h> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T03:23:44.560 DEBUG:teuthology.orchestra.run.smithi082:mds.k> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T03:23:44.603 DEBUG:teuthology.orchestra.run.smithi142:mds.c> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T03:23:44.606 DEBUG:teuthology.orchestra.run.smithi142:mds.f> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T03:23:44.649 DEBUG:teuthology.orchestra.run.smithi142:mds.i> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T03:23:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-12T03:23:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:44.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:44 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:44.693 DEBUG:teuthology.orchestra.run.smithi142:mds.l> sudo journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T03:23:44.735 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T03:23:44.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-12T03:23:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:44.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:44.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:44.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:44.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:44.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:44 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:44.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:44.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:44.881 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:44.882 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:44 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:45.104 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.626+0000 7f213c000640 1 -- 172.21.15.50:0/3428480094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134109120 msgr2=0x7f2134110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.626+0000 7f213c000640 1 --2- 172.21.15.50:0/3428480094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134109120 0x7f2134110ca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f212c009ca0 tx=0x7f212c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 -- 172.21.15.50:0/3428480094 shutdown_connections 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 --2- 172.21.15.50:0/3428480094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134109120 0x7f2134110ca0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 --2- 172.21.15.50:0/3428480094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21341049a0 0x7f21341089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 --2- 172.21.15.50:0/3428480094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2134103ff0 0x7f21341043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 -- 172.21.15.50:0/3428480094 >> 172.21.15.50:0/3428480094 conn(0x7f21340ffa90 msgr2=0x7f2134101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:45.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 -- 172.21.15.50:0/3428480094 shutdown_connections 2024-09-12T03:23:45.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.627+0000 7f213c000640 1 -- 172.21.15.50:0/3428480094 wait complete. 2024-09-12T03:23:45.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 Processor -- start 2024-09-12T03:23:45.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 -- start start 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134103ff0 0x7f213419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21341049a0 0x7f213419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2134109120 0x7f213419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21341141a0 con 0x7f2134103ff0 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2134114020 con 0x7f21341049a0 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f213c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2134114320 con 0x7f2134109120 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f2139600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134103ff0 0x7f213419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f2139600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134103ff0 0x7f213419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37234/0 (socket says 172.21.15.50:37234) 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.628+0000 7f2138c00640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21341049a0 0x7f213419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2139600640 1 -- 172.21.15.50:0/3815843409 learned_addr learned my addr 172.21.15.50:0/3815843409 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f213a000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2134109120 0x7f213419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2138c00640 1 -- 172.21.15.50:0/3815843409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2134109120 msgr2=0x7f213419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2138c00640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2134109120 0x7f213419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2138c00640 1 -- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134103ff0 msgr2=0x7f213419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2138c00640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134103ff0 0x7f213419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2138c00640 1 -- 172.21.15.50:0/3815843409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213419f590 con 0x7f21341049a0 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2138c00640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21341049a0 0x7f213419ab20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f212000e9f0 tx=0x7f212000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.629+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212000ccc0 con 0x7f21341049a0 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.630+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213419f880 con 0x7f21341049a0 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.630+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2120005020 con 0x7f21341049a0 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.630+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21200049e0 con 0x7f21341049a0 2024-09-12T03:23:45.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.630+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21341a7160 con 0x7f21341049a0 2024-09-12T03:23:45.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.631+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21341043d0 con 0x7f21341049a0 2024-09-12T03:23:45.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.633+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2120005560 con 0x7f21341049a0 2024-09-12T03:23:45.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.634+0000 7f2132000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f211c0a2b80 0x7f211c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.634+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f21200ceb10 con 0x7f21341049a0 2024-09-12T03:23:45.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.634+0000 7f2139600640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f211c0a2b80 0x7f211c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.635+0000 7f2139600640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f211c0a2b80 0x7f211c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2128005fd0 tx=0x7f212800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.636+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21200051d0 con 0x7f21341049a0 2024-09-12T03:23:45.834 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:45 smithi142 ceph-mon[35329]: Deploying daemon mds.l on smithi142 2024-09-12T03:23:45.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:45 smithi082 ceph-mon[35046]: Deploying daemon mds.l on smithi142 2024-09-12T03:23:45.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.856+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f213419b940 con 0x7f21341049a0 2024-09-12T03:23:45.858 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:45 smithi050 ceph-mon[30378]: Deploying daemon mds.l on smithi142 2024-09-12T03:23:45.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.857+0000 7f2132000640 1 -- 172.21.15.50:0/3815843409 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f213419b940 con 0x7f21341049a0 2024-09-12T03:23:45.858 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:45.858 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":1,"btime":"2024-09-12T03:16:17:849478+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T03:23:45.858 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 1 2024-09-12T03:23:45.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.859+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f211c0a2b80 msgr2=0x7f211c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.859+0000 7f213c000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f211c0a2b80 0x7f211c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2128005fd0 tx=0x7f212800b4a0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.859+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21341049a0 msgr2=0x7f213419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.859+0000 7f213c000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21341049a0 0x7f213419ab20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f212000e9f0 tx=0x7f212000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 shutdown_connections 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f211c0a2b80 0x7f211c0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2134109120 0x7f213419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21341049a0 0x7f213419ab20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 --2- 172.21.15.50:0/3815843409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2134103ff0 0x7f213419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 >> 172.21.15.50:0/3815843409 conn(0x7f21340ffa90 msgr2=0x7f21341005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 shutdown_connections 2024-09-12T03:23:45.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:45.860+0000 7f213c000640 1 -- 172.21.15.50:0/3815843409 wait complete. 2024-09-12T03:23:46.225 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 systemd[1]: Starting Ceph mds.l for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:23:46.517 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 podman[65205]: 2024-09-12 03:23:46.127590694 +0000 UTC m=+0.015719716 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:23:46.517 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 podman[65205]: 2024-09-12 03:23:46.247438513 +0000 UTC m=+0.135567527 container create ad4f9db4c1cf488768538fb6efa891b8ebcd67a18863c19b374b05d0ac0cfd32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:23:46.517 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 podman[65205]: 2024-09-12 03:23:46.517076336 +0000 UTC m=+0.405205369 container init ad4f9db4c1cf488768538fb6efa891b8ebcd67a18863c19b374b05d0ac0cfd32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:23:46.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-mon[35329]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:46.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3815843409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:46.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6832/1399602993,v1:172.21.15.142:6833/1399602993] up:boot 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-mon[35329]: fsmap 1 up:standby 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 podman[65205]: 2024-09-12 03:23:46.520387233 +0000 UTC m=+0.408516251 container start ad4f9db4c1cf488768538fb6efa891b8ebcd67a18863c19b374b05d0ac0cfd32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-mds[65225]: -- 172.21.15.142:0/2517025418 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb9e167860 con 0x55bb9e144800 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l[65221]: starting mds.l at 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 bash[65205]: ad4f9db4c1cf488768538fb6efa891b8ebcd67a18863c19b374b05d0ac0cfd32 2024-09-12T03:23:46.794 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:23:46 smithi142 systemd[1]: Started Ceph mds.l for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:23:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:46 smithi082 ceph-mon[35046]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:46 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3815843409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:46 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6832/1399602993,v1:172.21.15.142:6833/1399602993] up:boot 2024-09-12T03:23:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:46 smithi082 ceph-mon[35046]: fsmap 1 up:standby 2024-09-12T03:23:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:46 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:23:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:46 smithi050 ceph-mon[30378]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:46 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3815843409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:46 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6832/1399602993,v1:172.21.15.142:6833/1399602993] up:boot 2024-09-12T03:23:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:46 smithi050 ceph-mon[30378]: fsmap 1 up:standby 2024-09-12T03:23:46.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:46 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:48.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:47 smithi050 ceph-mon[30378]: Deploying daemon mds.k on smithi082 2024-09-12T03:23:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:48.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:48.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:48.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:47 smithi142 ceph-mon[35329]: Deploying daemon mds.k on smithi082 2024-09-12T03:23:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:48.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:48.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:48.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:48.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:47 smithi082 ceph-mon[35046]: Deploying daemon mds.k on smithi082 2024-09-12T03:23:48.666 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:48 smithi082 ceph-mon[35046]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:48.854 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:48 smithi082 systemd[1]: Starting Ceph mds.k for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:23:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:48 smithi142 ceph-mon[35329]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:49.051 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:48 smithi050 ceph-mon[30378]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:49.228 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:48 smithi082 podman[63935]: 2024-09-12 03:23:48.875678791 +0000 UTC m=+0.015731122 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:23:49.229 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:48 smithi082 podman[63935]: 2024-09-12 03:23:48.984659205 +0000 UTC m=+0.124711531 container create 53d8cbcad9db2105efc95b49502400b083094632e7b3e4c13071ac16abe0a36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:23:49.229 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:49 smithi082 podman[63935]: 2024-09-12 03:23:49.229064482 +0000 UTC m=+0.369116809 container init 53d8cbcad9db2105efc95b49502400b083094632e7b3e4c13071ac16abe0a36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:23:49.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.598+0000 7f5460800640 1 -- 172.21.15.50:0/2295505442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458112a30 msgr2=0x7f5458114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.598+0000 7f5460800640 1 --2- 172.21.15.50:0/2295505442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458112a30 0x7f5458114e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5454009ca0 tx=0x7f545402a8b0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.598+0000 7f5460800640 1 -- 172.21.15.50:0/2295505442 shutdown_connections 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.599+0000 7f5460800640 1 --2- 172.21.15.50:0/2295505442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458112a30 0x7f5458114e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.599+0000 7f5460800640 1 --2- 172.21.15.50:0/2295505442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5458077040 0x7f54581124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.599+0000 7f5460800640 1 --2- 172.21.15.50:0/2295505442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5458076720 0x7f5458076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.599+0000 7f5460800640 1 -- 172.21.15.50:0/2295505442 >> 172.21.15.50:0/2295505442 conn(0x7f54580ffa90 msgr2=0x7f5458101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.599+0000 7f5460800640 1 -- 172.21.15.50:0/2295505442 shutdown_connections 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.599+0000 7f5460800640 1 -- 172.21.15.50:0/2295505442 wait complete. 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 Processor -- start 2024-09-12T03:23:49.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 -- start start 2024-09-12T03:23:49.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 0x7f54581a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5458077040 0x7f54581a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5458112a30 0x7f545819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54581185c0 con 0x7f5458077040 2024-09-12T03:23:49.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5458118440 con 0x7f5458076720 2024-09-12T03:23:49.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f5460800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5458118740 con 0x7f5458112a30 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f545de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 0x7f54581a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.600+0000 7f545de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 0x7f54581a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42962/0 (socket says 172.21.15.50:42962) 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 -- 172.21.15.50:0/1699096825 learned_addr learned my addr 172.21.15.50:0/1699096825 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545e800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5458112a30 0x7f545819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545d400640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5458077040 0x7f54581a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 -- 172.21.15.50:0/1699096825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5458112a30 msgr2=0x7f545819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5458112a30 0x7f545819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 -- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5458077040 msgr2=0x7f54581a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5458077040 0x7f54581a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 -- 172.21.15.50:0/1699096825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545819efa0 con 0x7f5458076720 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.601+0000 7f545de00640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 0x7f54581a46b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f544c00ea60 tx=0x7f544c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.602+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c00cde0 con 0x7f5458076720 2024-09-12T03:23:49.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.602+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545819f290 con 0x7f5458076720 2024-09-12T03:23:49.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.602+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544c0035f0 con 0x7f5458076720 2024-09-12T03:23:49.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.602+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c010690 con 0x7f5458076720 2024-09-12T03:23:49.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.602+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581ab580 con 0x7f5458076720 2024-09-12T03:23:49.605 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:49 smithi082 podman[63935]: 2024-09-12 03:23:49.231951265 +0000 UTC m=+0.372003593 container start 53d8cbcad9db2105efc95b49502400b083094632e7b3e4c13071ac16abe0a36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:23:49.605 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:49 smithi082 ceph-mds[63954]: -- 172.21.15.82:0/1210228448 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d787b3860 con 0x563d78791000 2024-09-12T03:23:49.605 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k[63950]: starting mds.k at 2024-09-12T03:23:49.605 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:49 smithi082 bash[63935]: 53d8cbcad9db2105efc95b49502400b083094632e7b3e4c13071ac16abe0a36f 2024-09-12T03:23:49.606 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:23:49 smithi082 systemd[1]: Started Ceph mds.k for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:23:49.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.603+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5458076b00 con 0x7f5458076720 2024-09-12T03:23:49.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.604+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f544c0037a0 con 0x7f5458076720 2024-09-12T03:23:49.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.606+0000 7f5452a00640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54400a2b80 0x7f54400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.606+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f544c0cf060 con 0x7f5458076720 2024-09-12T03:23:49.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.606+0000 7f545d400640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54400a2b80 0x7f54400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.607+0000 7f545d400640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54400a2b80 0x7f54400a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5448005e00 tx=0x7f5448005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.610+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f544c09c3c0 con 0x7f5458076720 2024-09-12T03:23:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.837+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f545819fda0 con 0x7f5458076720 2024-09-12T03:23:49.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.838+0000 7f5452a00640 1 -- 172.21.15.50:0/1699096825 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7f545819fda0 con 0x7f5458076720 2024-09-12T03:23:49.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:49.839 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"btime":"2024-09-12T03:23:49:693988+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:23:49.839 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 3 2024-09-12T03:23:49.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54400a2b80 msgr2=0x7f54400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54400a2b80 0x7f54400a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5448005e00 tx=0x7f5448005c50 comp rx=0 tx=0).stop 2024-09-12T03:23:49.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 msgr2=0x7f54581a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 0x7f54581a46b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f544c00ea60 tx=0x7f544c00ef20 comp rx=0 tx=0).stop 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 shutdown_connections 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54400a2b80 0x7f54400a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5458112a30 0x7f545819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5458077040 0x7f54581a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 --2- 172.21.15.50:0/1699096825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5458076720 0x7f54581a46b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.840+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 >> 172.21.15.50:0/1699096825 conn(0x7f54580ffa90 msgr2=0x7f5458101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.841+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 shutdown_connections 2024-09-12T03:23:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:49.841+0000 7f5460800640 1 -- 172.21.15.50:0/1699096825 wait complete. 2024-09-12T03:23:50.510 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: Deploying daemon mds.j on smithi050 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6832/196275210,v1:172.21.15.82:6833/196275210] up:boot 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: fsmap 2 up:standby 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:23:50.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:50 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1699096825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:50.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:50.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:50.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: Deploying daemon mds.j on smithi050 2024-09-12T03:23:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6832/196275210,v1:172.21.15.82:6833/196275210] up:boot 2024-09-12T03:23:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: fsmap 2 up:standby 2024-09-12T03:23:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:23:50.650 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:50 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1699096825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:50.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: Deploying daemon mds.j on smithi050 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6832/196275210,v1:172.21.15.82:6833/196275210] up:boot 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: fsmap 2 up:standby 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:23:50.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:50 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1699096825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:52.434 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:52 smithi050 ceph-mon[30378]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:52.434 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:52 smithi050 systemd[1]: Starting Ceph mds.j for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:23:52.548 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:52.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:52 smithi142 ceph-mon[35329]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:52.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:52 smithi082 ceph-mon[35046]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:53.052 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:52 smithi050 podman[69888]: 2024-09-12 03:23:52.66771939 +0000 UTC m=+0.033620305 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:23:53.052 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:52 smithi050 podman[69888]: 2024-09-12 03:23:52.770689379 +0000 UTC m=+0.136590278 container create c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:23:53.052 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:53 smithi050 podman[69888]: 2024-09-12 03:23:53.007040195 +0000 UTC m=+0.372941105 container init c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:23:53.052 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:53 smithi050 podman[69888]: 2024-09-12 03:23:53.013717891 +0000 UTC m=+0.379618801 container start c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:23:53.052 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:53 smithi050 ceph-mds[69932]: -- 172.21.15.50:0/1936403324 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630b2b6dc20 con 0x5630b2b4b000 2024-09-12T03:23:53.183 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:53.324 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:53 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j[69917]: starting mds.j at 2024-09-12T03:23:53.325 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:53 smithi050 bash[69888]: c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 2024-09-12T03:23:53.325 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:23:53 smithi050 systemd[1]: Started Ceph mds.j for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:23:53.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.806+0000 7f56ef200640 1 -- 172.21.15.50:0/284256274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8108f60 msgr2=0x7f56e8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.806+0000 7f56ef200640 1 --2- 172.21.15.50:0/284256274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8108f60 0x7f56e8110ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009ca0 tx=0x7f56e402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.806+0000 7f56ef200640 1 -- 172.21.15.50:0/284256274 shutdown_connections 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.806+0000 7f56ef200640 1 --2- 172.21.15.50:0/284256274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8108f60 0x7f56e8110ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.806+0000 7f56ef200640 1 --2- 172.21.15.50:0/284256274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81047e0 0x7f56e8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.807+0000 7f56ef200640 1 --2- 172.21.15.50:0/284256274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e8103e30 0x7f56e8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.807+0000 7f56ef200640 1 -- 172.21.15.50:0/284256274 >> 172.21.15.50:0/284256274 conn(0x7f56e80ff910 msgr2=0x7f56e8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.807+0000 7f56ef200640 1 -- 172.21.15.50:0/284256274 shutdown_connections 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.807+0000 7f56ef200640 1 -- 172.21.15.50:0/284256274 wait complete. 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 Processor -- start 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 -- start start 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 0x7f56e819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81047e0 0x7f56e819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e8108f60 0x7f56e819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8114010 con 0x7f56e8103e30 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8113e90 con 0x7f56e81047e0 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ef200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8114190 con 0x7f56e8108f60 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 0x7f56e819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 0x7f56e819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57370/0 (socket says 172.21.15.50:57370) 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ede00640 1 -- 172.21.15.50:0/4226457659 learned_addr learned my addr 172.21.15.50:0/4226457659 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.808+0000 7f56ed400640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81047e0 0x7f56e819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ee800640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e8108f60 0x7f56e819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ede00640 1 -- 172.21.15.50:0/4226457659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e8108f60 msgr2=0x7f56e819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ede00640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e8108f60 0x7f56e819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ede00640 1 -- 172.21.15.50:0/4226457659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81047e0 msgr2=0x7f56e819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ede00640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81047e0 0x7f56e819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ede00640 1 -- 172.21.15.50:0/4226457659 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e819f3b0 con 0x7f56e8103e30 2024-09-12T03:23:53.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ede00640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 0x7f56e819a400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00e9f0 tx=0x7f56dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00cd80 con 0x7f56e8103e30 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56dc005500 con 0x7f56e8103e30 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e819f6a0 con 0x7f56e8103e30 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc010690 con 0x7f56e8103e30 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.809+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e81a6f80 con 0x7f56e8103e30 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.811+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e8105310 con 0x7f56e8103e30 2024-09-12T03:23:53.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.813+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f56dc0046c0 con 0x7f56e8103e30 2024-09-12T03:23:53.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.814+0000 7f56e2a00640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56d00a2b80 0x7f56d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.814+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f56dc0cf070 con 0x7f56e8103e30 2024-09-12T03:23:53.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.814+0000 7f56ed400640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56d00a2b80 0x7f56d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.815+0000 7f56ed400640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56d00a2b80 0x7f56d00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56e80ffd30 tx=0x7f56d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:53.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:53.818+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56dc09c470 con 0x7f56e8103e30 2024-09-12T03:23:54.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.048+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56e810c200 con 0x7f56e8103e30 2024-09-12T03:23:54.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.049+0000 7f56e2a00640 1 -- 172.21.15.50:0/4226457659 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f56dc09bd70 con 0x7f56e8103e30 2024-09-12T03:23:54.051 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:54.051 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":4,"btime":"2024-09-12T03:23:53:382835+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:23:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 4 2024-09-12T03:23:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56d00a2b80 msgr2=0x7f56d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56d00a2b80 0x7f56d00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56e80ffd30 tx=0x7f56d8005f90 comp rx=0 tx=0).stop 2024-09-12T03:23:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 msgr2=0x7f56e819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 0x7f56e819a400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00e9f0 tx=0x7f56dc00edf0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 shutdown_connections 2024-09-12T03:23:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56d00a2b80 0x7f56d00a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56e8108f60 0x7f56e819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56e81047e0 0x7f56e819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 --2- 172.21.15.50:0/4226457659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56e8103e30 0x7f56e819a400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 >> 172.21.15.50:0/4226457659 conn(0x7f56e80ff910 msgr2=0x7f56e8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 shutdown_connections 2024-09-12T03:23:54.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:54.051+0000 7f56ef200640 1 -- 172.21.15.50:0/4226457659 wait complete. 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: Deploying daemon mds.d on smithi050 2024-09-12T03:23:54.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.50:6834/3398998523,v1:172.21.15.50:6835/3398998523] up:boot 2024-09-12T03:23:54.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: fsmap 3 up:standby 2024-09-12T03:23:54.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:23:54.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:54 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4226457659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:54.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:54.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: Deploying daemon mds.d on smithi050 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.50:6834/3398998523,v1:172.21.15.50:6835/3398998523] up:boot 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: fsmap 3 up:standby 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:23:54.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:54 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4226457659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:54.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:54.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: Deploying daemon mds.d on smithi050 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.50:6834/3398998523,v1:172.21.15.50:6835/3398998523] up:boot 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: fsmap 3 up:standby 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:23:54.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:54 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4226457659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:56.400 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:56 smithi050 ceph-mon[30378]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:56.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:56 smithi142 ceph-mon[35329]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:56.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:56 smithi082 ceph-mon[35046]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:56.665 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 systemd[1]: Starting Ceph mds.d for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:23:56.857 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:56.974 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 podman[70284]: 2024-09-12 03:23:56.61625779 +0000 UTC m=+0.027335927 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:23:56.974 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 podman[70284]: 2024-09-12 03:23:56.726274922 +0000 UTC m=+0.137353048 container create be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:23:56.974 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 podman[70284]: 2024-09-12 03:23:56.92938536 +0000 UTC m=+0.340463516 container init be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T03:23:56.974 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 podman[70284]: 2024-09-12 03:23:56.935922766 +0000 UTC m=+0.347000894 container start be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:23:57.255 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:57.394 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 ceph-mds[70321]: -- 172.21.15.50:0/661358156 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648b2fcdc20 con 0x5648b2fab000 2024-09-12T03:23:57.394 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:56 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d[70299]: starting mds.d at 2024-09-12T03:23:57.394 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:57 smithi050 bash[70284]: be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd 2024-09-12T03:23:57.395 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:23:57 smithi050 systemd[1]: Started Ceph mds.d for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 -- 172.21.15.50:0/3405350385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741019c0 msgr2=0x7f2774101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 --2- 172.21.15.50:0/3405350385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741019c0 0x7f2774101e20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2760009930 tx=0x7f276002a420 comp rx=0 tx=0).stop 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 -- 172.21.15.50:0/3405350385 shutdown_connections 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 --2- 172.21.15.50:0/3405350385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2774102360 0x7f277410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 --2- 172.21.15.50:0/3405350385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741019c0 0x7f2774101e20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 --2- 172.21.15.50:0/3405350385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27741079c0 0x7f2774107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 -- 172.21.15.50:0/3405350385 >> 172.21.15.50:0/3405350385 conn(0x7f27740fd740 msgr2=0x7f27740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 -- 172.21.15.50:0/3405350385 shutdown_connections 2024-09-12T03:23:57.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.964+0000 7f277b000640 1 -- 172.21.15.50:0/3405350385 wait complete. 2024-09-12T03:23:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.965+0000 7f277b000640 1 Processor -- start 2024-09-12T03:23:57.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.965+0000 7f277b000640 1 -- start start 2024-09-12T03:23:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 0x7f277419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2774102360 0x7f277419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741079c0 0x7f277419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2774111dd0 con 0x7f27741079c0 2024-09-12T03:23:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2774111c50 con 0x7f2774102360 2024-09-12T03:23:57.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2774111f50 con 0x7f27741019c0 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 0x7f277419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 0x7f277419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58548/0 (socket says 172.21.15.50:58548) 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 -- 172.21.15.50:0/716604137 learned_addr learned my addr 172.21.15.50:0/716604137 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779200640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2774102360 0x7f277419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277a600640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741079c0 0x7f277419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 -- 172.21.15.50:0/716604137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2774102360 msgr2=0x7f277419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2774102360 0x7f277419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 -- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741079c0 msgr2=0x7f277419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741079c0 0x7f277419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f2779c00640 1 -- 172.21.15.50:0/716604137 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277419f4a0 con 0x7f27741019c0 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.966+0000 7f277a600640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741079c0 0x7f277419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:57.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.967+0000 7f2779c00640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 0x7f277419a4f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9f0 tx=0x7f276800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.967+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276800cd80 con 0x7f27741019c0 2024-09-12T03:23:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.967+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277419f790 con 0x7f27741019c0 2024-09-12T03:23:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.967+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2768010910 con 0x7f27741019c0 2024-09-12T03:23:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.967+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27680188f0 con 0x7f27741019c0 2024-09-12T03:23:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.967+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27741a7070 con 0x7f27741019c0 2024-09-12T03:23:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.968+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2774101e20 con 0x7f27741019c0 2024-09-12T03:23:57.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.970+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2768010ac0 con 0x7f27741019c0 2024-09-12T03:23:57.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.971+0000 7f2772000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f275c0a2b80 0x7f275c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.972+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f27680cf560 con 0x7f27741019c0 2024-09-12T03:23:57.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.972+0000 7f2779200640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f275c0a2b80 0x7f275c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.972+0000 7f2779200640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f275c0a2b80 0x7f275c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2760006010 tx=0x7f2760000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:57.978+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f276809c8c0 con 0x7f27741019c0 2024-09-12T03:23:58.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.198+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27741a7360 con 0x7f27741019c0 2024-09-12T03:23:58.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.199+0000 7f2772000640 1 -- 172.21.15.50:0/716604137 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f276809c1c0 con 0x7f27741019c0 2024-09-12T03:23:58.200 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:23:58.200 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":5,"btime":"2024-09-12T03:23:57:175167+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-12T03:23:58.201 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 5 2024-09-12T03:23:58.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f275c0a2b80 msgr2=0x7f275c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f275c0a2b80 0x7f275c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2760006010 tx=0x7f2760000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 msgr2=0x7f277419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 0x7f277419a4f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9f0 tx=0x7f276800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 shutdown_connections 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f275c0a2b80 0x7f275c0a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27741079c0 0x7f277419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2774102360 0x7f277419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 --2- 172.21.15.50:0/716604137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27741019c0 0x7f277419a4f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.201+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 >> 172.21.15.50:0/716604137 conn(0x7f27740fd740 msgr2=0x7f277410d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.202+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 shutdown_connections 2024-09-12T03:23:58.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:23:58.202+0000 7f277b000640 1 -- 172.21.15.50:0/716604137 wait complete. 2024-09-12T03:23:58.215 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:58.215 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.215 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: Deploying daemon mds.e on smithi082 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.50:6836/3806772200,v1:172.21.15.50:6837/3806772200] up:boot 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: fsmap 4 up:standby 2024-09-12T03:23:58.216 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:58 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: Deploying daemon mds.e on smithi082 2024-09-12T03:23:58.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.50:6836/3806772200,v1:172.21.15.50:6837/3806772200] up:boot 2024-09-12T03:23:58.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: fsmap 4 up:standby 2024-09-12T03:23:58.545 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:58 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:23:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:23:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:23:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:23:58.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:23:58.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:58.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: Deploying daemon mds.e on smithi082 2024-09-12T03:23:58.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.50:6836/3806772200,v1:172.21.15.50:6837/3806772200] up:boot 2024-09-12T03:23:58.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: fsmap 4 up:standby 2024-09-12T03:23:58.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:58 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:23:59.296 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:23:59 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/716604137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:59.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:23:59 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/716604137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:59.354 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 systemd[1]: Starting Ceph mds.e for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:23:59.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:23:59 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/716604137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:59.734 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 podman[64249]: 2024-09-12 03:23:59.286354653 +0000 UTC m=+0.015631487 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:23:59.735 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 podman[64249]: 2024-09-12 03:23:59.398715307 +0000 UTC m=+0.127992126 container create ec81a90b9220f59ca8418989c7b5296da8f5ca37b52fd0077114ea3f1e61c445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:23:59.735 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 podman[64249]: 2024-09-12 03:23:59.734606101 +0000 UTC m=+0.463882920 container init ec81a90b9220f59ca8418989c7b5296da8f5ca37b52fd0077114ea3f1e61c445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:24:00.104 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 podman[64249]: 2024-09-12 03:23:59.737458073 +0000 UTC m=+0.466734899 container start ec81a90b9220f59ca8418989c7b5296da8f5ca37b52fd0077114ea3f1e61c445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T03:24:00.105 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 ceph-mds[64268]: -- 172.21.15.82:0/2120623790 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ae2fb9860 con 0x563ae2f96800 2024-09-12T03:24:00.105 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e[64264]: starting mds.e at 2024-09-12T03:24:00.105 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 bash[64249]: ec81a90b9220f59ca8418989c7b5296da8f5ca37b52fd0077114ea3f1e61c445 2024-09-12T03:24:00.105 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:23:59 smithi082 systemd[1]: Started Ceph mds.e for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:00.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:00.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:00.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:00.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:00 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:00.477 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:00.477 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.477 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.477 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.477 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:00.477 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:00.478 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:00 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:00.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:00 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:00.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:01.388 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:01 smithi050 ceph-mon[30378]: Deploying daemon mds.f on smithi142 2024-09-12T03:24:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:01 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6834/1268137141,v1:172.21.15.82:6835/1268137141] up:boot 2024-09-12T03:24:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:01 smithi050 ceph-mon[30378]: fsmap 5 up:standby 2024-09-12T03:24:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:01 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:24:01.460 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:01 smithi142 ceph-mon[35329]: Deploying daemon mds.f on smithi142 2024-09-12T03:24:01.460 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:01 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6834/1268137141,v1:172.21.15.82:6835/1268137141] up:boot 2024-09-12T03:24:01.460 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:01 smithi142 ceph-mon[35329]: fsmap 5 up:standby 2024-09-12T03:24:01.461 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:01 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:24:01.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:01 smithi082 ceph-mon[35046]: Deploying daemon mds.f on smithi142 2024-09-12T03:24:01.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:01 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6834/1268137141,v1:172.21.15.82:6835/1268137141] up:boot 2024-09-12T03:24:01.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:01 smithi082 ceph-mon[35046]: fsmap 5 up:standby 2024-09-12T03:24:01.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:01 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 -- 172.21.15.50:0/2173880343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e0103ff0 msgr2=0x7fb1e01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 --2- 172.21.15.50:0/2173880343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e0103ff0 0x7fb1e01043d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009900 tx=0x7fb1d002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 -- 172.21.15.50:0/2173880343 shutdown_connections 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 --2- 172.21.15.50:0/2173880343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0109120 0x7fb1e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 --2- 172.21.15.50:0/2173880343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e01049a0 0x7fb1e01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 --2- 172.21.15.50:0/2173880343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e0103ff0 0x7fb1e01043d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 -- 172.21.15.50:0/2173880343 >> 172.21.15.50:0/2173880343 conn(0x7fb1e00ffa90 msgr2=0x7fb1e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 -- 172.21.15.50:0/2173880343 shutdown_connections 2024-09-12T03:24:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.912+0000 7fb1e7600640 1 -- 172.21.15.50:0/2173880343 wait complete. 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.913+0000 7fb1e7600640 1 Processor -- start 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.913+0000 7fb1e7600640 1 -- start start 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.913+0000 7fb1e7600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0103ff0 0x7fb1e019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.913+0000 7fb1e7600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 0x7fb1e019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e7600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e0109120 0x7fb1e019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e7600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e0114170 con 0x7fb1e01049a0 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e7600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e0113ff0 con 0x7fb1e0103ff0 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e7600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01142f0 con 0x7fb1e0109120 2024-09-12T03:24:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 0x7fb1e019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0103ff0 0x7fb1e019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0103ff0 0x7fb1e019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34066/0 (socket says 172.21.15.50:34066) 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 0x7fb1e019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43178/0 (socket says 172.21.15.50:43178) 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e4c00640 1 -- 172.21.15.50:0/3595544338 learned_addr learned my addr 172.21.15.50:0/3595544338 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e5600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e0109120 0x7fb1e019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 -- 172.21.15.50:0/3595544338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e0109120 msgr2=0x7fb1e019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e0109120 0x7fb1e019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 -- 172.21.15.50:0/3595544338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0103ff0 msgr2=0x7fb1e019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0103ff0 0x7fb1e019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1d7e00640 1 -- 172.21.15.50:0/3595544338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e019f620 con 0x7fb1e01049a0 2024-09-12T03:24:01.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.914+0000 7fb1e5600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e0109120 0x7fb1e019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:01.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.915+0000 7fb1d7e00640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 0x7fb1e019abb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800e9f0 tx=0x7fb1c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.915+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800ccc0 con 0x7fb1e01049a0 2024-09-12T03:24:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.915+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c8010910 con 0x7fb1e01049a0 2024-09-12T03:24:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.915+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e019f910 con 0x7fb1e01049a0 2024-09-12T03:24:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.915+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8018960 con 0x7fb1e01049a0 2024-09-12T03:24:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.915+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e01a71f0 con 0x7fb1e01049a0 2024-09-12T03:24:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.916+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e01043d0 con 0x7fb1e01049a0 2024-09-12T03:24:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.918+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb1c8010ac0 con 0x7fb1e01049a0 2024-09-12T03:24:01.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.919+0000 7fb1de000640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb1c40a2b80 0x7fb1c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.919+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb1c80cefd0 con 0x7fb1e01049a0 2024-09-12T03:24:01.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.920+0000 7fb1e4c00640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb1c40a2b80 0x7fb1c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.920+0000 7fb1e4c00640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb1c40a2b80 0x7fb1c40a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00096c0 tx=0x7fb1d00046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:01.923+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1c809bce0 con 0x7fb1e01049a0 2024-09-12T03:24:02.135 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 podman[65517]: 2024-09-12 03:24:02.07697176 +0000 UTC m=+0.152054720 container create 1c1bbd01c1d1f0fba25054bf3a697b6745ee7360f66dcebfa30a910d4c1c1829 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:24:02.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.147+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1e010bd70 con 0x7fb1e01049a0 2024-09-12T03:24:02.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.147+0000 7fb1de000640 1 -- 172.21.15.50:0/3595544338 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7fb1c809ffe0 con 0x7fb1e01049a0 2024-09-12T03:24:02.148 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:02.148 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":6,"btime":"2024-09-12T03:24:00:123133+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T03:24:02.149 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 6 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb1c40a2b80 msgr2=0x7fb1c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb1c40a2b80 0x7fb1c40a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00096c0 tx=0x7fb1d00046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 msgr2=0x7fb1e019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 0x7fb1e019abb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800e9f0 tx=0x7fb1c800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 shutdown_connections 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb1c40a2b80 0x7fb1c40a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1e0109120 0x7fb1e019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1e01049a0 0x7fb1e019abb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 --2- 172.21.15.50:0/3595544338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1e0103ff0 0x7fb1e019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 >> 172.21.15.50:0/3595544338 conn(0x7fb1e00ffa90 msgr2=0x7fb1e0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 shutdown_connections 2024-09-12T03:24:02.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:02.150+0000 7fb1e7600640 1 -- 172.21.15.50:0/3595544338 wait complete. 2024-09-12T03:24:02.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:02 smithi050 ceph-mon[30378]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:02.429 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:02 smithi142 ceph-mon[35329]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:02.429 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 podman[65517]: 2024-09-12 03:24:02.321174513 +0000 UTC m=+0.396257474 container init 1c1bbd01c1d1f0fba25054bf3a697b6745ee7360f66dcebfa30a910d4c1c1829 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T03:24:02.429 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 podman[65517]: 2024-09-12 03:24:02.324016854 +0000 UTC m=+0.399099815 container start 1c1bbd01c1d1f0fba25054bf3a697b6745ee7360f66dcebfa30a910d4c1c1829 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T03:24:02.429 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 ceph-mds[65536]: -- 172.21.15.142:0/441270157 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9efcc1860 con 0x55b9efc9f400 2024-09-12T03:24:02.430 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[65532]: starting mds.f at 2024-09-12T03:24:02.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:02 smithi082 ceph-mon[35046]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:02.793 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 bash[65517]: 1c1bbd01c1d1f0fba25054bf3a697b6745ee7360f66dcebfa30a910d4c1c1829 2024-09-12T03:24:02.793 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:24:02 smithi142 systemd[1]: Started Ceph mds.f for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3595544338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:03 smithi050 ceph-mon[30378]: Deploying daemon mds.c on smithi142 2024-09-12T03:24:03.174 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3595544338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:03.174 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.174 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.174 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.175 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:03.175 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:03.175 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:03.175 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:03 smithi142 ceph-mon[35329]: Deploying daemon mds.c on smithi142 2024-09-12T03:24:03.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3595544338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:03.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:03.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:03.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:03.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:03.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:03 smithi082 ceph-mon[35046]: Deploying daemon mds.c on smithi142 2024-09-12T03:24:04.285 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:04 smithi142 ceph-mon[35329]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:04.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:04 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6834/809625574,v1:172.21.15.142:6835/809625574] up:boot 2024-09-12T03:24:04.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:04 smithi142 ceph-mon[35329]: fsmap 6 up:standby 2024-09-12T03:24:04.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:04 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:24:04.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:04 smithi050 ceph-mon[30378]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:04.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:04 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6834/809625574,v1:172.21.15.142:6835/809625574] up:boot 2024-09-12T03:24:04.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:04 smithi050 ceph-mon[30378]: fsmap 6 up:standby 2024-09-12T03:24:04.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:04 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:24:04.543 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:04 smithi142 systemd[1]: Starting Ceph mds.c for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:24:04.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:04 smithi082 ceph-mon[35046]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:04.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:04 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6834/809625574,v1:172.21.15.142:6835/809625574] up:boot 2024-09-12T03:24:04.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:04 smithi082 ceph-mon[35046]: fsmap 6 up:standby 2024-09-12T03:24:04.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:04 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:24:04.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:04.976 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:04 smithi142 podman[65818]: 2024-09-12 03:24:04.572147499 +0000 UTC m=+0.015835750 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:24:04.976 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:04 smithi142 podman[65818]: 2024-09-12 03:24:04.698781028 +0000 UTC m=+0.142469279 container create 6b794d63593af548cdf307ff3db49df87b49403353e14a0ef86a70498d25c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:24:04.976 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:04 smithi142 podman[65818]: 2024-09-12 03:24:04.976649345 +0000 UTC m=+0.420337596 container init 6b794d63593af548cdf307ff3db49df87b49403353e14a0ef86a70498d25c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:24:05.293 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:04 smithi142 podman[65818]: 2024-09-12 03:24:04.980391501 +0000 UTC m=+0.424079751 container start 6b794d63593af548cdf307ff3db49df87b49403353e14a0ef86a70498d25c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:24:05.293 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:05 smithi142 ceph-mds[65838]: -- 172.21.15.142:0/3253685133 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622fb713860 con 0x5622fb6f1000 2024-09-12T03:24:05.294 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c[65834]: starting mds.c at 2024-09-12T03:24:05.294 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:05 smithi142 bash[65818]: 6b794d63593af548cdf307ff3db49df87b49403353e14a0ef86a70498d25c064 2024-09-12T03:24:05.294 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:24:05 smithi142 systemd[1]: Started Ceph mds.c for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:05.330 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 -- 172.21.15.50:0/2565633215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1781027a0 msgr2=0x7fd1781067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 --2- 172.21.15.50:0/2565633215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1781027a0 0x7fd1781067f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 -- 172.21.15.50:0/2565633215 shutdown_connections 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 --2- 172.21.15.50:0/2565633215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd178106f20 0x7fd17810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 --2- 172.21.15.50:0/2565633215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1781027a0 0x7fd1781067f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 --2- 172.21.15.50:0/2565633215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd1781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 -- 172.21.15.50:0/2565633215 >> 172.21.15.50:0/2565633215 conn(0x7fd1780fd8b0 msgr2=0x7fd1780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 -- 172.21.15.50:0/2565633215 shutdown_connections 2024-09-12T03:24:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.813+0000 7fd17e600640 1 -- 172.21.15.50:0/2565633215 wait complete. 2024-09-12T03:24:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 Processor -- start 2024-09-12T03:24:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 -- start start 2024-09-12T03:24:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd17819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781027a0 0x7fd17819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178106f20 0x7fd17819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd178111fd0 con 0x7fd178106f20 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd178111e50 con 0x7fd178101df0 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.814+0000 7fd17e600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd178112150 con 0x7fd1781027a0 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd17819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd17819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34088/0 (socket says 172.21.15.50:34088) 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 -- 172.21.15.50:0/3790559255 learned_addr learned my addr 172.21.15.50:0/3790559255 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177e00640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178106f20 0x7fd17819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd16fe00640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781027a0 0x7fd17819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 -- 172.21.15.50:0/3790559255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781027a0 msgr2=0x7fd17819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781027a0 0x7fd17819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 -- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178106f20 msgr2=0x7fd17819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178106f20 0x7fd17819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 -- 172.21.15.50:0/3790559255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17819f5a0 con 0x7fd178101df0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177e00640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178106f20 0x7fd17819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.815+0000 7fd177400640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd17819a5f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd16800e9f0 tx=0x7fd16800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.816+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16800cd80 con 0x7fd178101df0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.816+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17819f890 con 0x7fd178101df0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.816+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd168010910 con 0x7fd178101df0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.816+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680188f0 con 0x7fd178101df0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.816+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a7170 con 0x7fd178101df0 2024-09-12T03:24:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.817+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1781021d0 con 0x7fd178101df0 2024-09-12T03:24:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.819+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd168010ac0 con 0x7fd178101df0 2024-09-12T03:24:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.821+0000 7fd174c00640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd15c0a2b80 0x7fd15c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.821+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fd1680cf690 con 0x7fd178101df0 2024-09-12T03:24:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.821+0000 7fd16fe00640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd15c0a2b80 0x7fd15c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.822+0000 7fd16fe00640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd15c0a2b80 0x7fd15c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd160005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:05.822+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd16809c9f0 con 0x7fd178101df0 2024-09-12T03:24:06.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.046+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1781a7340 con 0x7fd178101df0 2024-09-12T03:24:06.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.046+0000 7fd174c00640 1 -- 172.21.15.50:0/3790559255 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7fd16809c2f0 con 0x7fd178101df0 2024-09-12T03:24:06.047 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:06.047 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"btime":"2024-09-12T03:24:05:143512+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T03:24:06.048 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd15c0a2b80 msgr2=0x7fd15c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd15c0a2b80 0x7fd15c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd160005a20 comp rx=0 tx=0).stop 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 msgr2=0x7fd17819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd17819a5f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd16800e9f0 tx=0x7fd16800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 shutdown_connections 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd15c0a2b80 0x7fd15c0a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178106f20 0x7fd17819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1781027a0 0x7fd17819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 --2- 172.21.15.50:0/3790559255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd178101df0 0x7fd17819a5f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 >> 172.21.15.50:0/3790559255 conn(0x7fd1780fd8b0 msgr2=0x7fd1780fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 shutdown_connections 2024-09-12T03:24:06.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:06.049+0000 7fd17e600640 1 -- 172.21.15.50:0/3790559255 wait complete. 2024-09-12T03:24:06.408 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: Deploying daemon mds.h on smithi082 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] up:boot 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: fsmap 7 up:standby 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:24:06.409 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:06 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3790559255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:06.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: Deploying daemon mds.h on smithi082 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] up:boot 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: fsmap 7 up:standby 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:24:06.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:06 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3790559255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:06.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: Deploying daemon mds.h on smithi082 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] up:boot 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: fsmap 7 up:standby 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:24:06.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:06 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3790559255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:07.354 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 systemd[1]: Starting Ceph mds.h for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:24:07.631 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 podman[64559]: 2024-09-12 03:24:07.281491215 +0000 UTC m=+0.015902211 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:24:07.631 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 podman[64559]: 2024-09-12 03:24:07.404394612 +0000 UTC m=+0.138805608 container create fa8492535347c21f2cff0a2ad6cc88c7f1a016f62fb15bc32d3effab74b880eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:24:07.631 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 podman[64559]: 2024-09-12 03:24:07.632048308 +0000 UTC m=+0.366459297 container init fa8492535347c21f2cff0a2ad6cc88c7f1a016f62fb15bc32d3effab74b880eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:24:08.104 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 podman[64559]: 2024-09-12 03:24:07.634946731 +0000 UTC m=+0.369357721 container start fa8492535347c21f2cff0a2ad6cc88c7f1a016f62fb15bc32d3effab74b880eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:24:08.104 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 ceph-mds[64578]: -- 172.21.15.82:0/3839706355 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd4197b860 con 0x55bd41958800 2024-09-12T03:24:08.104 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h[64574]: starting mds.h at 2024-09-12T03:24:08.104 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 bash[64559]: fa8492535347c21f2cff0a2ad6cc88c7f1a016f62fb15bc32d3effab74b880eb 2024-09-12T03:24:08.104 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:24:07 smithi082 systemd[1]: Started Ceph mds.h for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:08.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:08.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:08.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:08.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:08 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:08.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:08 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:08.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:08 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:08.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:09.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:09 smithi050 ceph-mon[30378]: Deploying daemon mds.i on smithi142 2024-09-12T03:24:09.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:09 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6836/4279679346,v1:172.21.15.82:6837/4279679346] up:boot 2024-09-12T03:24:09.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:09 smithi050 ceph-mon[30378]: fsmap 8 up:standby 2024-09-12T03:24:09.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:09 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:24:09.434 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:09 smithi142 ceph-mon[35329]: Deploying daemon mds.i on smithi142 2024-09-12T03:24:09.434 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:09 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6836/4279679346,v1:172.21.15.82:6837/4279679346] up:boot 2024-09-12T03:24:09.434 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:09 smithi142 ceph-mon[35329]: fsmap 8 up:standby 2024-09-12T03:24:09.434 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:09 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:24:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:09 smithi082 ceph-mon[35046]: Deploying daemon mds.i on smithi142 2024-09-12T03:24:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:09 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6836/4279679346,v1:172.21.15.82:6837/4279679346] up:boot 2024-09-12T03:24:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:09 smithi082 ceph-mon[35046]: fsmap 8 up:standby 2024-09-12T03:24:09.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:09 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:24:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 -- 172.21.15.50:0/3372481642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720102790 msgr2=0x7f27201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 --2- 172.21.15.50:0/3372481642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720102790 0x7f27201067e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2708009900 tx=0x7f270802a410 comp rx=0 tx=0).stop 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 -- 172.21.15.50:0/3372481642 shutdown_connections 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 --2- 172.21.15.50:0/3372481642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720106f10 0x7f272010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 --2- 172.21.15.50:0/3372481642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720102790 0x7f27201067e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 --2- 172.21.15.50:0/3372481642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2720101de0 0x7f27201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 -- 172.21.15.50:0/3372481642 >> 172.21.15.50:0/3372481642 conn(0x7f27200fd8c0 msgr2=0x7f27200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 -- 172.21.15.50:0/3372481642 shutdown_connections 2024-09-12T03:24:09.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.721+0000 7f2727200640 1 -- 172.21.15.50:0/3372481642 wait complete. 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.722+0000 7f2727200640 1 Processor -- start 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.722+0000 7f2727200640 1 -- start start 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2727200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2720101de0 0x7f27200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2727200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 0x7f2720074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2727200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720106f10 0x7f2720074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2727200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111f70 con 0x7f2720102790 2024-09-12T03:24:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2727200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111df0 con 0x7f2720101de0 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2727200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27201120f0 con 0x7f2720106f10 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 0x7f2720074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 0x7f2720074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44882/0 (socket says 172.21.15.50:44882) 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 -- 172.21.15.50:0/309648852 learned_addr learned my addr 172.21.15.50:0/309648852 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f271fe00640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2720101de0 0x7f27200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2725200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720106f10 0x7f2720074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 -- 172.21.15.50:0/309648852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720106f10 msgr2=0x7f2720074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720106f10 0x7f2720074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 -- 172.21.15.50:0/309648852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2720101de0 msgr2=0x7f27200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2720101de0 0x7f27200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2717e00640 1 -- 172.21.15.50:0/309648852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27201a6950 con 0x7f2720102790 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.723+0000 7f2725200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720106f10 0x7f2720074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.724+0000 7f2717e00640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 0x7f2720074a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2708006010 tx=0x7f2708004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.724+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270803f030 con 0x7f2720102790 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.724+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27201a6be0 con 0x7f2720102790 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.724+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2708004440 con 0x7f2720102790 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.724+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a70c0 con 0x7f2720102790 2024-09-12T03:24:09.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.724+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708049060 con 0x7f2720102790 2024-09-12T03:24:09.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.725+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27201021c0 con 0x7f2720102790 2024-09-12T03:24:09.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.727+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2708042030 con 0x7f2720102790 2024-09-12T03:24:09.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.728+0000 7f271d600640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f27040a2b80 0x7f27040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.729+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f27080ef240 con 0x7f2720102790 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.729+0000 7f271fe00640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f27040a2b80 0x7f27040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.729+0000 7f271fe00640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f27040a2b80 0x7f27040a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2710009bd0 tx=0x7f2710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.733+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27080bc5a0 con 0x7f2720102790 2024-09-12T03:24:09.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.952+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f272002d7c0 con 0x7f2720102790 2024-09-12T03:24:09.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.953+0000 7f271d600640 1 -- 172.21.15.50:0/309648852 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7f27080bbea0 con 0x7f2720102790 2024-09-12T03:24:09.953 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:09.953 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"btime":"2024-09-12T03:24:08:128380+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T03:24:09.954 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f27040a2b80 msgr2=0x7f27040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f27040a2b80 0x7f27040a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2710009bd0 tx=0x7f2710005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 msgr2=0x7f2720074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 0x7f2720074a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2708006010 tx=0x7f2708004040 comp rx=0 tx=0).stop 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 shutdown_connections 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f27040a2b80 0x7f27040a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2720106f10 0x7f2720074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2720102790 0x7f2720074a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 --2- 172.21.15.50:0/309648852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2720101de0 0x7f27200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.955+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 >> 172.21.15.50:0/309648852 conn(0x7f27200fd8c0 msgr2=0x7f27200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:09.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.956+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 shutdown_connections 2024-09-12T03:24:09.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:09.956+0000 7f2727200640 1 -- 172.21.15.50:0/309648852 wait complete. 2024-09-12T03:24:10.142 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 podman[66122]: 2024-09-12 03:24:10.046492561 +0000 UTC m=+0.139327922 container create 6a9a1ad592c82259699d07d1ca4f4bffecc430699be75dd6fb5ff5fada353217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:24:10.345 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:10 smithi050 ceph-mon[30378]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:10.345 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:10 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/309648852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:10.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:10 smithi142 ceph-mon[35329]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:10.392 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:10 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/309648852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:10.393 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 podman[66122]: 2024-09-12 03:24:10.299482723 +0000 UTC m=+0.392318083 container init 6a9a1ad592c82259699d07d1ca4f4bffecc430699be75dd6fb5ff5fada353217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:24:10.393 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 podman[66122]: 2024-09-12 03:24:10.30254264 +0000 UTC m=+0.395378010 container start 6a9a1ad592c82259699d07d1ca4f4bffecc430699be75dd6fb5ff5fada353217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:24:10.393 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 ceph-mds[66141]: -- 172.21.15.142:0/368525422 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1b0aa3860 con 0x55e1b0a81000 2024-09-12T03:24:10.393 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i[66137]: starting mds.i at 2024-09-12T03:24:10.393 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 bash[66122]: 6a9a1ad592c82259699d07d1ca4f4bffecc430699be75dd6fb5ff5fada353217 2024-09-12T03:24:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:10 smithi082 ceph-mon[35046]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:10 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/309648852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:10.793 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:24:10 smithi142 systemd[1]: Started Ceph mds.i for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:11.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: Deploying daemon mds.g on smithi050 2024-09-12T03:24:11.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] up:boot 2024-09-12T03:24:11.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: fsmap 9 up:standby 2024-09-12T03:24:11.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:11 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:24:11.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: Deploying daemon mds.g on smithi050 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] up:boot 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: fsmap 9 up:standby 2024-09-12T03:24:11.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:11 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: Deploying daemon mds.g on smithi050 2024-09-12T03:24:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] up:boot 2024-09-12T03:24:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: fsmap 9 up:standby 2024-09-12T03:24:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:11 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:24:12.725 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:12.775 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:12 smithi050 ceph-mon[30378]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:12 smithi142 ceph-mon[35329]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:12 smithi082 ceph-mon[35046]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:13.161 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:12 smithi050 systemd[1]: Starting Ceph mds.g for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:24:13.414 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 podman[71065]: 2024-09-12 03:24:13.065871959 +0000 UTC m=+0.030987606 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:24:13.415 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 podman[71065]: 2024-09-12 03:24:13.163819096 +0000 UTC m=+0.128934743 container create 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:24:13.620 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:13.760 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 podman[71065]: 2024-09-12 03:24:13.441468984 +0000 UTC m=+0.406584661 container init 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T03:24:13.813 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 podman[71065]: 2024-09-12 03:24:13.447552949 +0000 UTC m=+0.412668597 container start 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:24:13.813 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 ceph-mds[71103]: -- 172.21.15.50:0/1215281322 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654d622b860 con 0x5654d6209000 2024-09-12T03:24:13.813 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g[71099]: starting mds.g at 2024-09-12T03:24:13.813 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 bash[71065]: 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 2024-09-12T03:24:13.813 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:24:13 smithi050 systemd[1]: Started Ceph mds.g for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:14.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 -- 172.21.15.50:0/1456303776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 msgr2=0x7ff0bc0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 --2- 172.21.15.50:0/1456303776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc0749a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009860 tx=0x7ff0ac02a3d0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 -- 172.21.15.50:0/1456303776 shutdown_connections 2024-09-12T03:24:14.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 --2- 172.21.15.50:0/1456303776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc10e430 0x7ff0bc110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 --2- 172.21.15.50:0/1456303776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc074ee0 0x7ff0bc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 --2- 172.21.15.50:0/1456303776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc0749a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 -- 172.21.15.50:0/1456303776 >> 172.21.15.50:0/1456303776 conn(0x7ff0bc0fd8b0 msgr2=0x7ff0bc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 -- 172.21.15.50:0/1456303776 shutdown_connections 2024-09-12T03:24:14.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.178+0000 7ff0c1600640 1 -- 172.21.15.50:0/1456303776 wait complete. 2024-09-12T03:24:14.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 Processor -- start 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 -- start start 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc074ee0 0x7ff0bc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 0x7ff0bc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc114190 con 0x7ff0bc10e430 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc114010 con 0x7ff0bc0745c0 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.179+0000 7ff0c1600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc114310 con 0x7ff0bc074ee0 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 0x7ff0bc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc074ee0 0x7ff0bc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:46764/0 (socket says 172.21.15.50:46764) 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 0x7ff0bc19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44920/0 (socket says 172.21.15.50:44920) 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0baa00640 1 -- 172.21.15.50:0/2091925657 learned_addr learned my addr 172.21.15.50:0/2091925657 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 -- 172.21.15.50:0/2091925657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 msgr2=0x7ff0bc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 -- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 msgr2=0x7ff0bc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 0x7ff0bc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 -- 172.21.15.50:0/2091925657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc19f600 con 0x7ff0bc074ee0 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0bb400640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 0x7ff0bc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:14.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.180+0000 7ff0b2000640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc074ee0 0x7ff0bc19ab90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005fd0 tx=0x7ff0a400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.181+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400cd00 con 0x7ff0bc074ee0 2024-09-12T03:24:14.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.181+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc19f8f0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.181+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a40035f0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.181+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4010690 con 0x7ff0bc074ee0 2024-09-12T03:24:14.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.181+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1a71d0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.182+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0bc0749a0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.184+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff0a40037a0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.186+0000 7ff0b3e00640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff0a00a2c00 0x7ff0a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.186+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff0a40ced90 con 0x7ff0bc074ee0 2024-09-12T03:24:14.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.186+0000 7ff0baa00640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff0a00a2c00 0x7ff0a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.187+0000 7ff0baa00640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff0a00a2c00 0x7ff0a00a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005e00 tx=0x7ff0ac0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.190+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0a4009510 con 0x7ff0bc074ee0 2024-09-12T03:24:14.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.429+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0bc02d7c0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.430+0000 7ff0b3e00640 1 -- 172.21.15.50:0/2091925657 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7ff0a409c0f0 con 0x7ff0bc074ee0 2024-09-12T03:24:14.430 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:14.430 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":10,"btime":"2024-09-12T03:24:11:142287+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:24:14.431 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 10 2024-09-12T03:24:14.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.433+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff0a00a2c00 msgr2=0x7ff0a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.433+0000 7ff0c1600640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff0a00a2c00 0x7ff0a00a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005e00 tx=0x7ff0ac0058b0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.433+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc074ee0 msgr2=0x7ff0bc19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.433+0000 7ff0c1600640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc074ee0 0x7ff0bc19ab90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005fd0 tx=0x7ff0a400ee50 comp rx=0 tx=0).stop 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 shutdown_connections 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff0a00a2c00 0x7ff0a00a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0bc10e430 0x7ff0bc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc074ee0 0x7ff0bc19ab90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 --2- 172.21.15.50:0/2091925657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0bc0745c0 0x7ff0bc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 >> 172.21.15.50:0/2091925657 conn(0x7ff0bc0fd8b0 msgr2=0x7ff0bc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 shutdown_connections 2024-09-12T03:24:14.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:14.434+0000 7ff0c1600640 1 -- 172.21.15.50:0/2091925657 wait complete. 2024-09-12T03:24:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: Deploying daemon mds.a on smithi050 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2091925657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.50:6838/3736856191,v1:172.21.15.50:6839/3736856191] up:boot 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: fsmap 10 up:standby 2024-09-12T03:24:14.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:24:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: Deploying daemon mds.a on smithi050 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2091925657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.50:6838/3736856191,v1:172.21.15.50:6839/3736856191] up:boot 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: fsmap 10 up:standby 2024-09-12T03:24:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: Deploying daemon mds.a on smithi050 2024-09-12T03:24:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2091925657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:15.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.50:6838/3736856191,v1:172.21.15.50:6839/3736856191] up:boot 2024-09-12T03:24:15.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: fsmap 10 up:standby 2024-09-12T03:24:15.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:24:16.870 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:16 smithi050 ceph-mon[30378]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:17.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:16 smithi142 ceph-mon[35329]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:16 smithi082 ceph-mon[35046]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:17.164 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:16 smithi050 systemd[1]: Starting Ceph mds.a for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:24:17.466 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 podman[71452]: 2024-09-12 03:24:17.089171602 +0000 UTC m=+0.027780908 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:24:17.466 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 podman[71452]: 2024-09-12 03:24:17.185708144 +0000 UTC m=+0.124317428 container create 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:24:17.466 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 podman[71452]: 2024-09-12 03:24:17.422005622 +0000 UTC m=+0.360614928 container init 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:24:17.466 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 podman[71452]: 2024-09-12 03:24:17.428350389 +0000 UTC m=+0.366959675 container start 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-09-12T03:24:17.466 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 ceph-mds[71472]: -- 172.21.15.50:0/2765699505 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b2b6452680 con 0x55b2b6427000 2024-09-12T03:24:17.466 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 ceph-mds[71472]: -- 172.21.15.50:0/2765699505 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2b6449c20 con 0x55b2b6427000 2024-09-12T03:24:17.761 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:17.811 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a[71468]: starting mds.a at 2024-09-12T03:24:17.811 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 bash[71452]: 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 2024-09-12T03:24:17.811 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:24:17 smithi050 systemd[1]: Started Ceph mds.a for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:18.142 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:18.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 -- 172.21.15.50:0/467821205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c1047e0 msgr2=0x7fe23c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 --2- 172.21.15.50:0/467821205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c1047e0 0x7fe23c108830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe224009900 tx=0x7fe22402a410 comp rx=0 tx=0).stop 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 -- 172.21.15.50:0/467821205 shutdown_connections 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 --2- 172.21.15.50:0/467821205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe23c108f60 0x7fe23c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 --2- 172.21.15.50:0/467821205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c1047e0 0x7fe23c108830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 --2- 172.21.15.50:0/467821205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c103e30 0x7fe23c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 -- 172.21.15.50:0/467821205 >> 172.21.15.50:0/467821205 conn(0x7fe23c0ff8d0 msgr2=0x7fe23c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 -- 172.21.15.50:0/467821205 shutdown_connections 2024-09-12T03:24:18.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.748+0000 7fe241000640 1 -- 172.21.15.50:0/467821205 wait complete. 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.749+0000 7fe241000640 1 Processor -- start 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 -- start start 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 0x7fe23c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1047e0 0x7fe23c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe23c108f60 0x7fe23c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c113fc0 con 0x7fe23c103e30 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c113e40 con 0x7fe23c1047e0 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe241000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c114140 con 0x7fe23c108f60 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe23b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 0x7fe23c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe23b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 0x7fe23c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44948/0 (socket says 172.21.15.50:44948) 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe23b400640 1 -- 172.21.15.50:0/1241435317 learned_addr learned my addr 172.21.15.50:0/1241435317 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe232a00640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1047e0 0x7fe23c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.750+0000 7fe23be00640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe23c108f60 0x7fe23c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe23b400640 1 -- 172.21.15.50:0/1241435317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe23c108f60 msgr2=0x7fe23c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe23b400640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe23c108f60 0x7fe23c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe23b400640 1 -- 172.21.15.50:0/1241435317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1047e0 msgr2=0x7fe23c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe23b400640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1047e0 0x7fe23c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe23b400640 1 -- 172.21.15.50:0/1241435317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c19f410 con 0x7fe23c103e30 2024-09-12T03:24:18.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe23b400640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 0x7fe23c19a460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00e9f0 tx=0x7fe22c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00cd80 con 0x7fe23c103e30 2024-09-12T03:24:18.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c010910 con 0x7fe23c103e30 2024-09-12T03:24:18.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.751+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c19f700 con 0x7fe23c103e30 2024-09-12T03:24:18.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.752+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c018990 con 0x7fe23c103e30 2024-09-12T03:24:18.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.752+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c1a6fe0 con 0x7fe23c103e30 2024-09-12T03:24:18.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.753+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c104210 con 0x7fe23c103e30 2024-09-12T03:24:18.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.755+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe22c010ac0 con 0x7fe23c103e30 2024-09-12T03:24:18.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.756+0000 7fe238c00640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe2200a2b80 0x7fe2200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.756+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fe22c0cef50 con 0x7fe23c103e30 2024-09-12T03:24:18.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.756+0000 7fe232a00640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe2200a2b80 0x7fe2200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.757+0000 7fe232a00640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe2200a2b80 0x7fe2200a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe224009230 tx=0x7fe2240058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.760+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe22c09bc60 con 0x7fe23c103e30 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:18.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:18.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: Deploying daemon mds.b on smithi082 2024-09-12T03:24:18.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.50:6840/2806488600,v1:172.21.15.50:6841/2806488600] up:boot 2024-09-12T03:24:18.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: fsmap 11 up:standby 2024-09-12T03:24:18.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:18 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:24:18.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.980+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe23c10c140 con 0x7fe23c103e30 2024-09-12T03:24:18.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.981+0000 7fe238c00640 1 -- 172.21.15.50:0/1241435317 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7fe22c09ff60 con 0x7fe23c103e30 2024-09-12T03:24:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:18.982 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":12,"btime":"2024-09-12T03:24:17:636880+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:24:18.983 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 12 2024-09-12T03:24:18.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.983+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe2200a2b80 msgr2=0x7fe2200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.983+0000 7fe241000640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe2200a2b80 0x7fe2200a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe224009230 tx=0x7fe2240058d0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.983+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 msgr2=0x7fe23c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.983+0000 7fe241000640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 0x7fe23c19a460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00e9f0 tx=0x7fe22c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 shutdown_connections 2024-09-12T03:24:18.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe2200a2b80 0x7fe2200a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe23c108f60 0x7fe23c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1047e0 0x7fe23c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 --2- 172.21.15.50:0/1241435317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe23c103e30 0x7fe23c19a460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 >> 172.21.15.50:0/1241435317 conn(0x7fe23c0ff8d0 msgr2=0x7fe23c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 shutdown_connections 2024-09-12T03:24:18.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:18.984+0000 7fe241000640 1 -- 172.21.15.50:0/1241435317 wait complete. 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: Deploying daemon mds.b on smithi082 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.50:6840/2806488600,v1:172.21.15.50:6841/2806488600] up:boot 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: fsmap 11 up:standby 2024-09-12T03:24:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:18 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:24:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:19.071 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:24:19.072 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:24:19.072 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:19.072 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: Deploying daemon mds.b on smithi082 2024-09-12T03:24:19.072 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.50:6840/2806488600,v1:172.21.15.50:6841/2806488600] up:boot 2024-09-12T03:24:19.072 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: fsmap 11 up:standby 2024-09-12T03:24:19.072 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:18 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:24:19.618 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:19 smithi082 systemd[1]: Starting Ceph mds.b for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:24:19.893 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:19 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1241435317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:19.893 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:19 smithi082 podman[64862]: 2024-09-12 03:24:19.657535792 +0000 UTC m=+0.015640048 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:24:19.893 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:19 smithi082 podman[64862]: 2024-09-12 03:24:19.774219135 +0000 UTC m=+0.132323422 container create 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:24:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:19 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1241435317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:20.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:19 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1241435317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:20.354 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:20 smithi082 podman[64862]: 2024-09-12 03:24:20.018624994 +0000 UTC m=+0.376729241 container init 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:24:20.354 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:20 smithi082 podman[64862]: 2024-09-12 03:24:20.021531241 +0000 UTC m=+0.379635485 container start 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:24:20.354 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mds[64881]: -- 172.21.15.82:0/3755302889 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56143bfbd860 con 0x56143bf9b400 2024-09-12T03:24:20.354 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: starting mds.b at 2024-09-12T03:24:20.354 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:20 smithi082 bash[64862]: 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 2024-09-12T03:24:20.354 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:20 smithi082 systemd[1]: Started Ceph mds.b for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:20 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:20.970 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:20.970 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.970 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.970 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.970 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.970 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.971 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:20 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:20.984 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:20.984 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.985 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.985 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.985 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.985 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:20.985 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:20 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:21.742 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:21 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] up:boot 2024-09-12T03:24:21.742 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:21 smithi082 ceph-mon[35046]: fsmap 12 up:standby 2024-09-12T03:24:21.742 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:21 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:24:21.852 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:21.905 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:21 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] up:boot 2024-09-12T03:24:21.905 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:21 smithi050 ceph-mon[30378]: fsmap 12 up:standby 2024-09-12T03:24:21.905 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:21 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:24:22.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:21 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] up:boot 2024-09-12T03:24:22.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:21 smithi142 ceph-mon[35329]: fsmap 12 up:standby 2024-09-12T03:24:22.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:21 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:24:22.290 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:22.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:22 smithi050 ceph-mon[30378]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:22.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:22.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:22.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:22.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:22 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 -- 172.21.15.50:0/479737446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b44109120 msgr2=0x7f9b44110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 --2- 172.21.15.50:0/479737446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b44109120 0x7f9b44110ca0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009ca0 tx=0x7f9b3c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 -- 172.21.15.50:0/479737446 shutdown_connections 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 --2- 172.21.15.50:0/479737446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b44109120 0x7f9b44110ca0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 --2- 172.21.15.50:0/479737446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b441049a0 0x7f9b441089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 --2- 172.21.15.50:0/479737446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44103ff0 0x7f9b441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 -- 172.21.15.50:0/479737446 >> 172.21.15.50:0/479737446 conn(0x7f9b440ffa90 msgr2=0x7f9b44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 -- 172.21.15.50:0/479737446 shutdown_connections 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.869+0000 7f9b49600640 1 -- 172.21.15.50:0/479737446 wait complete. 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.870+0000 7f9b49600640 1 Processor -- start 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.870+0000 7f9b49600640 1 -- start start 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.870+0000 7f9b49600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b44103ff0 0x7f9b4419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.870+0000 7f9b49600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 0x7f9b4419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b49600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44109120 0x7f9b4419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b49600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44114150 con 0x7f9b441049a0 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b49600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44113fd0 con 0x7f9b44109120 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b49600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b441142d0 con 0x7f9b44103ff0 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 0x7f9b4419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 0x7f9b4419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40262/0 (socket says 172.21.15.50:40262) 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 -- 172.21.15.50:0/2452842544 learned_addr learned my addr 172.21.15.50:0/2452842544 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b43400640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44109120 0x7f9b4419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42a00640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b44103ff0 0x7f9b4419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 -- 172.21.15.50:0/2452842544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b44103ff0 msgr2=0x7f9b4419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b44103ff0 0x7f9b4419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 -- 172.21.15.50:0/2452842544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44109120 msgr2=0x7f9b4419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44109120 0x7f9b4419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b42000640 1 -- 172.21.15.50:0/2452842544 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4419f560 con 0x7f9b441049a0 2024-09-12T03:24:22.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.871+0000 7f9b43400640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44109120 0x7f9b4419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.872+0000 7f9b42000640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 0x7f9b4419aaf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000e9f0 tx=0x7f9b3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.872+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000cd80 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.872+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b30005500 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.872+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4419f850 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.872+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30010690 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.872+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b441a7130 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.873+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b441043d0 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.875+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9b300046c0 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.877+0000 7f9b3b400640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9b280a2b80 0x7f9b280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.877+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f9b300cf1c0 con 0x7f9b441049a0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.877+0000 7f9b42a00640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9b280a2b80 0x7f9b280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.877+0000 7f9b42a00640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9b280a2b80 0x7f9b280a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009bd0 tx=0x7f9b34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:22.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:22.883+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b300541b0 con 0x7f9b441049a0 2024-09-12T03:24:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:22 smithi142 ceph-mon[35329]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:22 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:22 smithi082 ceph-mon[35046]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:22 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.106+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b4419b910 con 0x7f9b441049a0 2024-09-12T03:24:23.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.107+0000 7f9b3b400640 1 -- 172.21.15.50:0/2452842544 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f9b4419b910 con 0x7f9b441049a0 2024-09-12T03:24:23.108 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:23.108 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":13,"btime":"2024-09-12T03:24:20:619325+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T03:24:23.108 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 13 2024-09-12T03:24:23.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9b280a2b80 msgr2=0x7f9b280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:23.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9b280a2b80 0x7f9b280a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009bd0 tx=0x7f9b34005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:23.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 msgr2=0x7f9b4419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:23.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 0x7f9b4419aaf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000e9f0 tx=0x7f9b3000edf0 comp rx=0 tx=0).stop 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 shutdown_connections 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f9b280a2b80 0x7f9b280a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44109120 0x7f9b4419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b441049a0 0x7f9b4419aaf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 --2- 172.21.15.50:0/2452842544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b44103ff0 0x7f9b4419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.110+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 >> 172.21.15.50:0/2452842544 conn(0x7f9b440ffa90 msgr2=0x7f9b44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.111+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 shutdown_connections 2024-09-12T03:24:23.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:23.111+0000 7f9b49600640 1 -- 172.21.15.50:0/2452842544 wait complete. 2024-09-12T03:24:23.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2452842544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:23.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:23.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:23.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:23.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:23 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:23.833 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2452842544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:23 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2452842544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:23 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:24.898 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:24 smithi142 ceph-mon[35329]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:25.044 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:24 smithi050 ceph-mon[30378]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:24 smithi082 ceph-mon[35046]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:25.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 -- 172.21.15.50:0/1681363770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 msgr2=0x7fddfc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 --2- 172.21.15.50:0/1681363770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 0x7fddfc1043d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fddec009900 tx=0x7fddec02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 -- 172.21.15.50:0/1681363770 shutdown_connections 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 --2- 172.21.15.50:0/1681363770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc109120 0x7fddfc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 --2- 172.21.15.50:0/1681363770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddfc1049a0 0x7fddfc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 --2- 172.21.15.50:0/1681363770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 0x7fddfc1043d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 -- 172.21.15.50:0/1681363770 >> 172.21.15.50:0/1681363770 conn(0x7fddfc0ffa90 msgr2=0x7fddfc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.612+0000 7fde01400640 1 -- 172.21.15.50:0/1681363770 shutdown_connections 2024-09-12T03:24:25.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.613+0000 7fde01400640 1 -- 172.21.15.50:0/1681363770 wait complete. 2024-09-12T03:24:25.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.613+0000 7fde01400640 1 Processor -- start 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 -- start start 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 0x7fddfc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 0x7fddfc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddfc109120 0x7fddfc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc114150 con 0x7fddfc103ff0 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc113fd0 con 0x7fddfc109120 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.614+0000 7fde01400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1142d0 con 0x7fddfc1049a0 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 0x7fddfc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddfa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 0x7fddfc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddfb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddfc109120 0x7fddfc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 0x7fddfc19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:54076/0 (socket says 172.21.15.50:54076) 2024-09-12T03:24:25.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 -- 172.21.15.50:0/3439952103 learned_addr learned my addr 172.21.15.50:0/3439952103 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 -- 172.21.15.50:0/3439952103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddfc109120 msgr2=0x7fddfc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddfc109120 0x7fddfc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 -- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 msgr2=0x7fddfc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 0x7fddfc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.615+0000 7fddf3e00640 1 -- 172.21.15.50:0/3439952103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc19f5a0 con 0x7fddfc1049a0 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.616+0000 7fddf3e00640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 0x7fddfc19ab30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e9f0 tx=0x7fdde400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:25.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.616+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde400cd80 con 0x7fddfc1049a0 2024-09-12T03:24:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.616+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc19f890 con 0x7fddfc1049a0 2024-09-12T03:24:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.617+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1a7170 con 0x7fddfc1049a0 2024-09-12T03:24:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.617+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde4010910 con 0x7fddfc1049a0 2024-09-12T03:24:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.617+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4018980 con 0x7fddfc1049a0 2024-09-12T03:24:25.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.618+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd8005290 con 0x7fddfc1049a0 2024-09-12T03:24:25.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.621+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdde4010ac0 con 0x7fddfc1049a0 2024-09-12T03:24:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.622+0000 7fddf3400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdde00a2c00 0x7fdde00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.623+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fdde40ceff0 con 0x7fddfc1049a0 2024-09-12T03:24:25.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.623+0000 7fddfa800640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdde00a2c00 0x7fdde00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.623+0000 7fddfa800640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdde00a2c00 0x7fdde00a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fddec009230 tx=0x7fddec0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:25.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.624+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdde409bd00 con 0x7fddfc1049a0 2024-09-12T03:24:25.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.831+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fddd8005850 con 0x7fddfc1049a0 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.832+0000 7fddf3400640 1 -- 172.21.15.50:0/3439952103 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fdde40a0000 con 0x7fddfc1049a0 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:24:25.833 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-09-12T03:24:25.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.834+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdde00a2c00 msgr2=0x7fdde00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.834+0000 7fde01400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdde00a2c00 0x7fdde00a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fddec009230 tx=0x7fddec0058d0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.834+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 msgr2=0x7fddfc19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.834+0000 7fde01400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 0x7fddfc19ab30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e9f0 tx=0x7fdde400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 shutdown_connections 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdde00a2c00 0x7fdde00a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddfc109120 0x7fddfc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddfc1049a0 0x7fddfc19ab30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 --2- 172.21.15.50:0/3439952103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddfc103ff0 0x7fddfc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 >> 172.21.15.50:0/3439952103 conn(0x7fddfc0ffa90 msgr2=0x7fddfc100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 shutdown_connections 2024-09-12T03:24:25.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:25.835+0000 7fde01400640 1 -- 172.21.15.50:0/3439952103 wait complete. 2024-09-12T03:24:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:25 smithi050 ceph-mon[30378]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:25 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:25 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:25 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:25 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:25 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3439952103' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:24:26.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:25 smithi142 ceph-mon[35329]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:26.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:25 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:25 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:25 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:25 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:25 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3439952103' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:24:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:25 smithi082 ceph-mon[35046]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:25 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:25 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:25 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:25 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:25 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3439952103' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:24:26.567 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:24:26.568 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:24:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 -- 172.21.15.50:0/1241266726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0106f10 msgr2=0x7f04f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 --2- 172.21.15.50:0/1241266726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0106f10 0x7f04f010ea90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009900 tx=0x7f04e802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 -- 172.21.15.50:0/1241266726 shutdown_connections 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 --2- 172.21.15.50:0/1241266726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0106f10 0x7f04f010ea90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 --2- 172.21.15.50:0/1241266726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102790 0x7f04f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 --2- 172.21.15.50:0/1241266726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0101de0 0x7f04f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 -- 172.21.15.50:0/1241266726 >> 172.21.15.50:0/1241266726 conn(0x7f04f00fd8c0 msgr2=0x7f04f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 -- 172.21.15.50:0/1241266726 shutdown_connections 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.517+0000 7f04f4c00640 1 -- 172.21.15.50:0/1241266726 wait complete. 2024-09-12T03:24:27.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 Processor -- start 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 -- start start 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 0x7f04f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0102790 0x7f04f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0106f10 0x7f04f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0111fc0 con 0x7f04f0101de0 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0111e40 con 0x7f04f0106f10 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04f4c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0112140 con 0x7f04f0102790 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04ede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 0x7f04f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04ede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 0x7f04f00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40302/0 (socket says 172.21.15.50:40302) 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.518+0000 7f04ede00640 1 -- 172.21.15.50:0/4022560727 learned_addr learned my addr 172.21.15.50:0/4022560727 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ed400640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0102790 0x7f04f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ee800640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0106f10 0x7f04f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ede00640 1 -- 172.21.15.50:0/4022560727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0102790 msgr2=0x7f04f0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ede00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0102790 0x7f04f0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ede00640 1 -- 172.21.15.50:0/4022560727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0106f10 msgr2=0x7f04f0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ede00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0106f10 0x7f04f0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ede00640 1 -- 172.21.15.50:0/4022560727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f01a68f0 con 0x7f04f0101de0 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ee800640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0106f10 0x7f04f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04ede00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 0x7f04f00744f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f04e000ea80 tx=0x7f04e000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e000ce40 con 0x7f04f0101de0 2024-09-12T03:24:27.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e0005020 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0011760 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.519+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01a6be0 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.520+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a70d0 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.521+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f01021c0 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.523+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f04e00051d0 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.525+0000 7f04e6a00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f04d40a2b80 0x7f04d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.525+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f04e00cf280 con 0x7f04f0101de0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.525+0000 7f04ed400640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f04d40a2b80 0x7f04d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.526+0000 7f04ed400640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f04d40a2b80 0x7f04d40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f04f01087b0 tx=0x7f04dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:27.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.528+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04e0016030 con 0x7f04f0101de0 2024-09-12T03:24:27.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:27.711+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f04f010a0f0 con 0x7f04f0101de0 2024-09-12T03:24:28.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.288+0000 7f04e6a00640 1 -- 172.21.15.50:0/4022560727 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f04e009c5e0 con 0x7f04f0101de0 2024-09-12T03:24:28.289 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-09-12T03:24:28.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.291+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f04d40a2b80 msgr2=0x7f04d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:28.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.291+0000 7f04f4c00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f04d40a2b80 0x7f04d40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f04f01087b0 tx=0x7f04dc005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:28.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.291+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 msgr2=0x7f04f00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:28.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.291+0000 7f04f4c00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 0x7f04f00744f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f04e000ea80 tx=0x7f04e000ef40 comp rx=0 tx=0).stop 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 shutdown_connections 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f04d40a2b80 0x7f04d40a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0106f10 0x7f04f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04f0102790 0x7f04f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 --2- 172.21.15.50:0/4022560727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04f0101de0 0x7f04f00744f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 >> 172.21.15.50:0/4022560727 conn(0x7f04f00fd8c0 msgr2=0x7f04f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 shutdown_connections 2024-09-12T03:24:28.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:28.292+0000 7f04f4c00640 1 -- 172.21.15.50:0/4022560727 wait complete. 2024-09-12T03:24:28.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:28.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:28.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:28 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:28.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:28.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:28 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.614 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.614 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.614 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:28.614 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:28.614 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:28 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:24:28.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-12T03:24:29.350 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:29.506 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:29 smithi050 ceph-mon[30378]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:29.506 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:29 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4022560727' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:24:29.506 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:29 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4022560727' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:24:29.506 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:29 smithi050 ceph-mon[30378]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:24:29.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:29 smithi142 ceph-mon[35329]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:29.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:29 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4022560727' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:24:29.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:29 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4022560727' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:24:29.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:29 smithi142 ceph-mon[35329]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:24:29.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:29 smithi082 ceph-mon[35046]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:29.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:29 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4022560727' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:24:29.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:29 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4022560727' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:24:29.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:29 smithi082 ceph-mon[35046]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.885+0000 7f8c7fe00640 1 -- 172.21.15.50:0/350178236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109140 msgr2=0x7f8c78110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.885+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/350178236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109140 0x7f8c78110cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009ca0 tx=0x7f8c7002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 -- 172.21.15.50:0/350178236 shutdown_connections 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/350178236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109140 0x7f8c78110cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/350178236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c781049c0 0x7f8c78108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/350178236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 0x7f8c781043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 -- 172.21.15.50:0/350178236 >> 172.21.15.50:0/350178236 conn(0x7f8c780ffa50 msgr2=0x7f8c78101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 -- 172.21.15.50:0/350178236 shutdown_connections 2024-09-12T03:24:29.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.886+0000 7f8c7fe00640 1 -- 172.21.15.50:0/350178236 wait complete. 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 Processor -- start 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 -- start start 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 0x7f8c7819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 0x7f8c7819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c78109140 0x7f8c7819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781141c0 con 0x7f8c781049c0 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78114040 con 0x7f8c78104010 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78114340 con 0x7f8c78109140 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.887+0000 7f8c7ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 0x7f8c7819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 0x7f8c7819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60892/0 (socket says 172.21.15.50:60892) 2024-09-12T03:24:29.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7ca00640 1 -- 172.21.15.50:0/3591110503 learned_addr learned my addr 172.21.15.50:0/3591110503 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 0x7f8c7819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7de00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c78109140 0x7f8c7819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 -- 172.21.15.50:0/3591110503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c78109140 msgr2=0x7f8c7819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c78109140 0x7f8c7819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 -- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 msgr2=0x7f8c7819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 0x7f8c7819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 -- 172.21.15.50:0/3591110503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7819f650 con 0x7f8c78104010 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7ca00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 0x7f8c7819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:29.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.888+0000 7f8c7d400640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 0x7f8c7819a6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00e9f0 tx=0x7f8c6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.889+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00ccc0 con 0x7f8c78104010 2024-09-12T03:24:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.889+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6c010910 con 0x7f8c78104010 2024-09-12T03:24:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.889+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c018960 con 0x7f8c78104010 2024-09-12T03:24:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.889+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7819f940 con 0x7f8c78104010 2024-09-12T03:24:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.889+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a7220 con 0x7f8c78104010 2024-09-12T03:24:29.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.890+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c781043f0 con 0x7f8c78104010 2024-09-12T03:24:29.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.892+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c6c010ac0 con 0x7f8c78104010 2024-09-12T03:24:29.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.893+0000 7f8c76000640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8c600a2b80 0x7f8c600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.893+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7497+0+0 (secure 0 0 0) 0x7f8c6c0cf140 con 0x7f8c78104010 2024-09-12T03:24:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.893+0000 7f8c7ca00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8c600a2b80 0x7f8c600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:29.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.894+0000 7f8c7ca00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8c600a2b80 0x7f8c600a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c7810aab0 tx=0x7f8c64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:29.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:29.897+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c6c09bce0 con 0x7f8c78104010 2024-09-12T03:24:30.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.081+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f8c781a7470 con 0x7f8c78104010 2024-09-12T03:24:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.620+0000 7f8c76000640 1 -- 172.21.15.50:0/3591110503 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) ==== 171+0+0 (secure 0 0 0) 0x7f8c6c09ffe0 con 0x7f8c78104010 2024-09-12T03:24:30.621 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.625+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8c600a2b80 msgr2=0x7f8c600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.625+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8c600a2b80 0x7f8c600a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c7810aab0 tx=0x7f8c64005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.625+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 msgr2=0x7f8c7819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.625+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 0x7f8c7819a6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00e9f0 tx=0x7f8c6c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 shutdown_connections 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8c600a2b80 0x7f8c600a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c78109140 0x7f8c7819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c781049c0 0x7f8c7819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 --2- 172.21.15.50:0/3591110503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c78104010 0x7f8c7819a6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:30.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 >> 172.21.15.50:0/3591110503 conn(0x7f8c780ffa50 msgr2=0x7f8c78101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:30.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 shutdown_connections 2024-09-12T03:24:30.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:30.626+0000 7f8c7fe00640 1 -- 172.21.15.50:0/3591110503 wait complete. 2024-09-12T03:24:30.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:30 smithi050 ceph-mon[30378]: pgmap v287: 33 pgs: 11 creating+peering, 1 active+clean, 21 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:30.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:30 smithi050 ceph-mon[30378]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:24:30.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:30 smithi050 ceph-mon[30378]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:24:30.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:30 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3591110503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:24:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:30 smithi142 ceph-mon[35329]: pgmap v287: 33 pgs: 11 creating+peering, 1 active+clean, 21 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:30 smithi142 ceph-mon[35329]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:24:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:30 smithi142 ceph-mon[35329]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:24:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:30 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3591110503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:24:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:30 smithi082 ceph-mon[35046]: pgmap v287: 33 pgs: 11 creating+peering, 1 active+clean, 21 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:30 smithi082 ceph-mon[35046]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:24:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:30 smithi082 ceph-mon[35046]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:24:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:30 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3591110503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:24:31.443 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:24:31.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:31 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:24:31.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:31 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:24:31.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:31 smithi050 ceph-mon[30378]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:24:31.836 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:32.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:31 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:24:32.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:31 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:24:32.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:31 smithi142 ceph-mon[35329]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:24:32.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:31 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:24:32.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:31 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:24:32.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:31 smithi082 ceph-mon[35046]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 -- 172.21.15.50:0/1374587734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198076720 msgr2=0x7f4198076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 --2- 172.21.15.50:0/1374587734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198076720 0x7f4198076b00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4188009900 tx=0x7f418802a410 comp rx=0 tx=0).stop 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 -- 172.21.15.50:0/1374587734 shutdown_connections 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 --2- 172.21.15.50:0/1374587734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 0x7f4198114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 --2- 172.21.15.50:0/1374587734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198077040 0x7f41981124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 --2- 172.21.15.50:0/1374587734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198076720 0x7f4198076b00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 -- 172.21.15.50:0/1374587734 >> 172.21.15.50:0/1374587734 conn(0x7f41980ffa90 msgr2=0x7f4198101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:32.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 -- 172.21.15.50:0/1374587734 shutdown_connections 2024-09-12T03:24:32.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.391+0000 7f419ee00640 1 -- 172.21.15.50:0/1374587734 wait complete. 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.392+0000 7f419ee00640 1 Processor -- start 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.392+0000 7f419ee00640 1 -- start start 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198076720 0x7f41981a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198077040 0x7f41981a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 0x7f419819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f41981185a0 con 0x7f4198112a30 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4198118420 con 0x7f4198077040 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4198118720 con 0x7f4198076720 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f4197e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198076720 0x7f41981a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f419ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 0x7f419819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f4197e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198076720 0x7f41981a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56030/0 (socket says 172.21.15.50:56030) 2024-09-12T03:24:32.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f4197e00640 1 -- 172.21.15.50:0/693104836 learned_addr learned my addr 172.21.15.50:0/693104836 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:32.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.393+0000 7f418f400640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198077040 0x7f41981a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:32.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ce00640 1 -- 172.21.15.50:0/693104836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198076720 msgr2=0x7f41981a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:32.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ce00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198076720 0x7f41981a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:32.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ce00640 1 -- 172.21.15.50:0/693104836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198077040 msgr2=0x7f41981a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ce00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198077040 0x7f41981a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ce00640 1 -- 172.21.15.50:0/693104836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419819efc0 con 0x7f4198112a30 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ce00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 0x7f419819e890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f419807aa10 tx=0x7f419000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419000cdf0 con 0x7f4198112a30 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419819f2b0 con 0x7f4198112a30 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4190005020 con 0x7f4198112a30 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190010670 con 0x7f4198112a30 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.394+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981ab5a0 con 0x7f4198112a30 2024-09-12T03:24:32.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.395+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4198076b00 con 0x7f4198112a30 2024-09-12T03:24:32.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.397+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f41900054d0 con 0x7f4198112a30 2024-09-12T03:24:32.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.400+0000 7f4195600640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f417c0a2b80 0x7f417c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:32.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.400+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7f41900cf530 con 0x7f4198112a30 2024-09-12T03:24:32.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.400+0000 7f4197e00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f417c0a2b80 0x7f417c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:32.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.401+0000 7f4197e00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f417c0a2b80 0x7f417c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41880096c0 tx=0x7f4188005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:32.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.402+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f419009c5a0 con 0x7f4198112a30 2024-09-12T03:24:32.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:32.625+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f419810d630 con 0x7f4198112a30 2024-09-12T03:24:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:32 smithi050 ceph-mon[30378]: pgmap v290: 97 pgs: 17 creating+peering, 10 active+clean, 70 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:32 smithi050 ceph-mon[30378]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:24:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:32 smithi142 ceph-mon[35329]: pgmap v290: 97 pgs: 17 creating+peering, 10 active+clean, 70 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:32 smithi142 ceph-mon[35329]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:24:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:32 smithi082 ceph-mon[35046]: pgmap v290: 97 pgs: 17 creating+peering, 10 active+clean, 70 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:32 smithi082 ceph-mon[35046]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:24:33.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.633+0000 7f4195600640 1 -- 172.21.15.50:0/693104836 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f419009bea0 con 0x7f4198112a30 2024-09-12T03:24:33.634 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:24:33.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.636+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f417c0a2b80 msgr2=0x7f417c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.636+0000 7f419ee00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f417c0a2b80 0x7f417c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41880096c0 tx=0x7f4188005a20 comp rx=0 tx=0).stop 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.636+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 msgr2=0x7f419819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.636+0000 7f419ee00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 0x7f419819e890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f419807aa10 tx=0x7f419000ef10 comp rx=0 tx=0).stop 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 shutdown_connections 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f417c0a2b80 0x7f417c0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4198112a30 0x7f419819e890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4198077040 0x7f41981a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 --2- 172.21.15.50:0/693104836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4198076720 0x7f41981a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 >> 172.21.15.50:0/693104836 conn(0x7f41980ffa90 msgr2=0x7f4198101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 shutdown_connections 2024-09-12T03:24:33.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:33.637+0000 7f419ee00640 1 -- 172.21.15.50:0/693104836 wait complete. 2024-09-12T03:24:33.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:33 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/693104836' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:24:33.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:33 smithi050 ceph-mon[30378]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:24:33.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:33 smithi050 ceph-mon[30378]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:24:33.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:33 smithi050 ceph-mon[30378]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:24:34.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:33 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/693104836' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:24:34.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:33 smithi142 ceph-mon[35329]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:24:34.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:33 smithi142 ceph-mon[35329]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:24:34.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:33 smithi142 ceph-mon[35329]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:24:34.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:33 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/693104836' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:24:34.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:33 smithi082 ceph-mon[35046]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:24:34.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:33 smithi082 ceph-mon[35046]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:24:34.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:33 smithi082 ceph-mon[35046]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:24:34.438 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-12T03:24:34.439 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-12T03:24:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:34 smithi050 ceph-mon[30378]: pgmap v294: 97 pgs: 26 creating+peering, 13 active+clean, 58 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:34 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/693104836' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:24:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:34 smithi050 ceph-mon[30378]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:24:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:34 smithi050 ceph-mon[30378]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:24:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:34 smithi050 ceph-mon[30378]: Cluster is now healthy 2024-09-12T03:24:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:34 smithi050 ceph-mon[30378]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:24:34.821 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:34 smithi142 ceph-mon[35329]: pgmap v294: 97 pgs: 26 creating+peering, 13 active+clean, 58 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:34 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/693104836' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:24:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:34 smithi142 ceph-mon[35329]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:24:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:34 smithi142 ceph-mon[35329]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:24:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:34 smithi142 ceph-mon[35329]: Cluster is now healthy 2024-09-12T03:24:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:34 smithi142 ceph-mon[35329]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:24:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:34 smithi082 ceph-mon[35046]: pgmap v294: 97 pgs: 26 creating+peering, 13 active+clean, 58 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:34 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/693104836' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:24:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:34 smithi082 ceph-mon[35046]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:24:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:34 smithi082 ceph-mon[35046]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:24:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:34 smithi082 ceph-mon[35046]: Cluster is now healthy 2024-09-12T03:24:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:34 smithi082 ceph-mon[35046]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:24:35.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 -- 172.21.15.50:0/680656315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c109140 msgr2=0x7f054c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:35.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 --2- 172.21.15.50:0/680656315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c109140 0x7f054c110cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0544009ca0 tx=0x7f054402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 -- 172.21.15.50:0/680656315 shutdown_connections 2024-09-12T03:24:35.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 --2- 172.21.15.50:0/680656315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c109140 0x7f054c110cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 --2- 172.21.15.50:0/680656315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c1049c0 0x7f054c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 --2- 172.21.15.50:0/680656315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054c104010 0x7f054c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 -- 172.21.15.50:0/680656315 >> 172.21.15.50:0/680656315 conn(0x7f054c0ffa50 msgr2=0x7f054c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:35.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 -- 172.21.15.50:0/680656315 shutdown_connections 2024-09-12T03:24:35.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.362+0000 7f0552200640 1 -- 172.21.15.50:0/680656315 wait complete. 2024-09-12T03:24:35.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 Processor -- start 2024-09-12T03:24:35.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 -- start start 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 0x7f054c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c1049c0 0x7f054c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054c109140 0x7f054c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f054c114170 con 0x7f054c1049c0 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f054c113ff0 con 0x7f054c104010 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.363+0000 7f0552200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f054c1142f0 con 0x7f054c109140 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 0x7f054c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c1049c0 0x7f054c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 0x7f054c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:60832/0 (socket says 172.21.15.50:60832) 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 -- 172.21.15.50:0/1858817168 learned_addr learned my addr 172.21.15.50:0/1858817168 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:35.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054be00640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054c109140 0x7f054c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 -- 172.21.15.50:0/1858817168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054c109140 msgr2=0x7f054c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054c109140 0x7f054c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 -- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c1049c0 msgr2=0x7f054c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c1049c0 0x7f054c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 -- 172.21.15.50:0/1858817168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c19f5c0 con 0x7f054c104010 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054aa00640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c1049c0 0x7f054c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.364+0000 7f054b400640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 0x7f054c19a610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f053c00e9f0 tx=0x7f053c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.365+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c00cd80 con 0x7f054c104010 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.365+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c005500 con 0x7f054c104010 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.365+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c19f8b0 con 0x7f054c104010 2024-09-12T03:24:35.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.365+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c010690 con 0x7f054c104010 2024-09-12T03:24:35.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.365+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c1a7190 con 0x7f054c104010 2024-09-12T03:24:35.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.366+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c1043f0 con 0x7f054c104010 2024-09-12T03:24:35.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.368+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f053c0046c0 con 0x7f054c104010 2024-09-12T03:24:35.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.369+0000 7f0543e00640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05300a2b80 0x7f05300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:35.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.369+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f053c0cf370 con 0x7f054c104010 2024-09-12T03:24:35.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.369+0000 7f054aa00640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05300a2b80 0x7f05300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:35.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.370+0000 7f054aa00640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05300a2b80 0x7f05300a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f054c10ab10 tx=0x7f0538005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:35.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.373+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f053c09c380 con 0x7f054c104010 2024-09-12T03:24:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.568+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f054c10c3e0 con 0x7f054c104010 2024-09-12T03:24:35.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.660+0000 7f0543e00640 1 -- 172.21.15.50:0/1858817168 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f053c09bc80 con 0x7f054c104010 2024-09-12T03:24:35.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05300a2b80 msgr2=0x7f05300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:35.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05300a2b80 0x7f05300a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f054c10ab10 tx=0x7f0538005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:35.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 msgr2=0x7f054c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:35.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 0x7f054c19a610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f053c00e9f0 tx=0x7f053c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 shutdown_connections 2024-09-12T03:24:35.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05300a2b80 0x7f05300a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054c109140 0x7f054c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f054c1049c0 0x7f054c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 --2- 172.21.15.50:0/1858817168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f054c104010 0x7f054c19a610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:35.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 >> 172.21.15.50:0/1858817168 conn(0x7f054c0ffa50 msgr2=0x7f054c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:35.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 shutdown_connections 2024-09-12T03:24:35.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:35.662+0000 7f0552200640 1 -- 172.21.15.50:0/1858817168 wait complete. 2024-09-12T03:24:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:35 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] up:active 2024-09-12T03:24:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:35 smithi050 ceph-mon[30378]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:35 smithi050 ceph-mon[30378]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:35 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1858817168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:24:35.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:35 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:24:36.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:35 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] up:active 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:35 smithi142 ceph-mon[35329]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:35 smithi142 ceph-mon[35329]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:35 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1858817168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:35 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:24:36.104 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:24:35 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: 2024-09-12T03:24:35.632+0000 7fe019f8e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:24:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:35 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] up:active 2024-09-12T03:24:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:35 smithi082 ceph-mon[35046]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:35 smithi082 ceph-mon[35046]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:36.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:35 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1858817168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:24:36.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:35 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:24:36.553 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:24:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:36 smithi050 ceph-mon[30378]: pgmap v295: 97 pgs: 9 creating+peering, 84 active+clean, 4 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s wr, 6 op/s 2024-09-12T03:24:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:36 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:24:36.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:36 smithi050 ceph-mon[30378]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:24:36.935 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:36 smithi142 ceph-mon[35329]: pgmap v295: 97 pgs: 9 creating+peering, 84 active+clean, 4 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s wr, 6 op/s 2024-09-12T03:24:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:36 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:24:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:36 smithi142 ceph-mon[35329]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:24:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:36 smithi082 ceph-mon[35046]: pgmap v295: 97 pgs: 9 creating+peering, 84 active+clean, 4 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s wr, 6 op/s 2024-09-12T03:24:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:36 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:24:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:36 smithi082 ceph-mon[35046]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:24:37.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 -- 172.21.15.50:0/1123856432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed301047e0 msgr2=0x7fed30108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:37.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 --2- 172.21.15.50:0/1123856432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed301047e0 0x7fed30108830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009900 tx=0x7fed1c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:37.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 -- 172.21.15.50:0/1123856432 shutdown_connections 2024-09-12T03:24:37.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 --2- 172.21.15.50:0/1123856432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30108f60 0x7fed30110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:37.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 --2- 172.21.15.50:0/1123856432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed301047e0 0x7fed30108830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:37.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 --2- 172.21.15.50:0/1123856432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed30103e30 0x7fed30104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:37.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 -- 172.21.15.50:0/1123856432 >> 172.21.15.50:0/1123856432 conn(0x7fed300ff910 msgr2=0x7fed30101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:37.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 -- 172.21.15.50:0/1123856432 shutdown_connections 2024-09-12T03:24:37.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.514+0000 7fed36800640 1 -- 172.21.15.50:0/1123856432 wait complete. 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 Processor -- start 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 -- start start 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 0x7fed3019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed301047e0 0x7fed3019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed30108f60 0x7fed3019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed30113fc0 con 0x7fed30108f60 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fed30113e40 con 0x7fed301047e0 2024-09-12T03:24:37.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.515+0000 7fed36800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed30114140 con 0x7fed30103e30 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 0x7fed3019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed34a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed301047e0 0x7fed3019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 0x7fed3019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56050/0 (socket says 172.21.15.50:56050) 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 -- 172.21.15.50:0/3964828684 learned_addr learned my addr 172.21.15.50:0/3964828684 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35e00640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed30108f60 0x7fed3019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 -- 172.21.15.50:0/3964828684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed301047e0 msgr2=0x7fed3019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed301047e0 0x7fed3019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 -- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed30108f60 msgr2=0x7fed3019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed30108f60 0x7fed3019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.516+0000 7fed35400640 1 -- 172.21.15.50:0/3964828684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3019f460 con 0x7fed30103e30 2024-09-12T03:24:37.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.517+0000 7fed35400640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 0x7fed3019a4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed2400d950 tx=0x7fed2400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.517+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24017070 con 0x7fed30103e30 2024-09-12T03:24:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.517+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24009c30 con 0x7fed30103e30 2024-09-12T03:24:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.517+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3019f750 con 0x7fed30103e30 2024-09-12T03:24:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.517+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2400f6b0 con 0x7fed30103e30 2024-09-12T03:24:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.517+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301a7030 con 0x7fed30103e30 2024-09-12T03:24:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.518+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed30105310 con 0x7fed30103e30 2024-09-12T03:24:37.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.520+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed240109d0 con 0x7fed30103e30 2024-09-12T03:24:37.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.521+0000 7fed2e000640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fed180a2b80 0x7fed180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:37.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.522+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7fed240cec40 con 0x7fed30103e30 2024-09-12T03:24:37.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.522+0000 7fed34a00640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fed180a2b80 0x7fed180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:37.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.522+0000 7fed34a00640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fed180a2b80 0x7fed180a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009230 tx=0x7fed1c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:37.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.524+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed24009de0 con 0x7fed30103e30 2024-09-12T03:24:37.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:37.709+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fed30104210 con 0x7fed30103e30 2024-09-12T03:24:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:38 smithi050 ceph-mon[30378]: pgmap v297: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-12T03:24:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:38 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3964828684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:24:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:38 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:24:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:38 smithi142 ceph-mon[35329]: pgmap v297: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-12T03:24:39.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:38 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3964828684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:24:39.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:38 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:24:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:38 smithi082 ceph-mon[35046]: pgmap v297: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-12T03:24:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:38 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3964828684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:24:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:38 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:24:39.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.680+0000 7fed2e000640 1 -- 172.21.15.50:0/3964828684 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7fed2409b4c0 con 0x7fed30103e30 2024-09-12T03:24:39.683 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' created 2024-09-12T03:24:39.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.686+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fed180a2b80 msgr2=0x7fed180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:39.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.686+0000 7fed36800640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fed180a2b80 0x7fed180a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009230 tx=0x7fed1c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:24:39.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.686+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 msgr2=0x7fed3019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:39.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.686+0000 7fed36800640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 0x7fed3019a4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed2400d950 tx=0x7fed2400de10 comp rx=0 tx=0).stop 2024-09-12T03:24:39.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 shutdown_connections 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fed180a2b80 0x7fed180a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed30108f60 0x7fed3019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed301047e0 0x7fed3019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 --2- 172.21.15.50:0/3964828684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed30103e30 0x7fed3019a4b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 >> 172.21.15.50:0/3964828684 conn(0x7fed300ff910 msgr2=0x7fed30100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 shutdown_connections 2024-09-12T03:24:39.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:39.687+0000 7fed36800640 1 -- 172.21.15.50:0/3964828684 wait complete. 2024-09-12T03:24:40.023 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:39 smithi050 ceph-mon[30378]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:24:40.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:39 smithi142 ceph-mon[35329]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:24:40.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:39 smithi082 ceph-mon[35046]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:24:40.321 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-12T03:24:40.723 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:41.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:40 smithi050 ceph-mon[30378]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-12T03:24:41.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:40 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:24:41.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:40 smithi050 ceph-mon[30378]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:24:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:40 smithi142 ceph-mon[35329]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-12T03:24:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:40 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:24:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:40 smithi142 ceph-mon[35329]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:24:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:40 smithi082 ceph-mon[35046]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-12T03:24:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:40 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:24:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:40 smithi082 ceph-mon[35046]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.258+0000 7fe37d000640 1 -- 172.21.15.50:0/1128629543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378109140 msgr2=0x7fe378110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.258+0000 7fe37d000640 1 --2- 172.21.15.50:0/1128629543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378109140 0x7fe378110cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe370009ca0 tx=0x7fe37002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 -- 172.21.15.50:0/1128629543 shutdown_connections 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 --2- 172.21.15.50:0/1128629543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378109140 0x7fe378110cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 --2- 172.21.15.50:0/1128629543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 0x7fe378108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 --2- 172.21.15.50:0/1128629543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe378104010 0x7fe3781043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 -- 172.21.15.50:0/1128629543 >> 172.21.15.50:0/1128629543 conn(0x7fe3780ffa50 msgr2=0x7fe378101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 -- 172.21.15.50:0/1128629543 shutdown_connections 2024-09-12T03:24:41.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 -- 172.21.15.50:0/1128629543 wait complete. 2024-09-12T03:24:41.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.259+0000 7fe37d000640 1 Processor -- start 2024-09-12T03:24:41.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 -- start start 2024-09-12T03:24:41.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 0x7fe37819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 0x7fe37819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe378109140 0x7fe37819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe378114150 con 0x7fe378104010 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe378113fd0 con 0x7fe3781049c0 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe37d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781142d0 con 0x7fe378109140 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe375e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 0x7fe37819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe375e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 0x7fe37819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46162/0 (socket says 172.21.15.50:46162) 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe375e00640 1 -- 172.21.15.50:0/4222248055 learned_addr learned my addr 172.21.15.50:0/4222248055 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe36d400640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 0x7fe37819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.260+0000 7fe376800640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe378109140 0x7fe37819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe375e00640 1 -- 172.21.15.50:0/4222248055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe378109140 msgr2=0x7fe37819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe375e00640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe378109140 0x7fe37819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe375e00640 1 -- 172.21.15.50:0/4222248055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 msgr2=0x7fe37819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe375e00640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 0x7fe37819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe375e00640 1 -- 172.21.15.50:0/4222248055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37819f5a0 con 0x7fe378104010 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe36d400640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 0x7fe37819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.261+0000 7fe375e00640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 0x7fe37819a5f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e9f0 tx=0x7fe36800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.262+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800cd80 con 0x7fe378104010 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.262+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368010910 con 0x7fe378104010 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.262+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37819f890 con 0x7fe378104010 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.262+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368018990 con 0x7fe378104010 2024-09-12T03:24:41.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.262+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781a7170 con 0x7fe378104010 2024-09-12T03:24:41.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.263+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3781043f0 con 0x7fe378104010 2024-09-12T03:24:41.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.265+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe368010ac0 con 0x7fe378104010 2024-09-12T03:24:41.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.267+0000 7fe36f400640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe35c0a2b80 0x7fe35c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.267+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fe3680cfa10 con 0x7fe378104010 2024-09-12T03:24:41.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.267+0000 7fe36d400640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe35c0a2b80 0x7fe35c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.268+0000 7fe36d400640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe35c0a2b80 0x7fe35c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe37810ab10 tx=0x7fe360005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:41.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.269+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe36809c0a0 con 0x7fe378104010 2024-09-12T03:24:41.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.452+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe37819b950 con 0x7fe378104010 2024-09-12T03:24:41.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.691+0000 7fe36f400640 1 -- 172.21.15.50:0/4222248055 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fe3680a03a0 con 0x7fe378104010 2024-09-12T03:24:41.693 INFO:teuthology.orchestra.run.smithi050.stderr:set pool 4 allow_ec_overwrites to true 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe35c0a2b80 msgr2=0x7fe35c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe35c0a2b80 0x7fe35c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe37810ab10 tx=0x7fe360005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 msgr2=0x7fe37819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 0x7fe37819a5f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e9f0 tx=0x7fe36800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 shutdown_connections 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe35c0a2b80 0x7fe35c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe378109140 0x7fe37819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3781049c0 0x7fe37819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 --2- 172.21.15.50:0/4222248055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe378104010 0x7fe37819a5f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 >> 172.21.15.50:0/4222248055 conn(0x7fe3780ffa50 msgr2=0x7fe378100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 shutdown_connections 2024-09-12T03:24:41.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:41.695+0000 7fe37d000640 1 -- 172.21.15.50:0/4222248055 wait complete. 2024-09-12T03:24:41.710 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:41 smithi050 ceph-mon[30378]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:24:41.710 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:41 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4222248055' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:24:42.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:41 smithi142 ceph-mon[35329]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:24:42.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:41 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4222248055' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:24:42.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:41 smithi082 ceph-mon[35046]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:24:42.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:41 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4222248055' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:24:42.384 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-12T03:24:42.762 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:42 smithi142 ceph-mon[35329]: pgmap v302: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-12T03:24:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:42 smithi142 ceph-mon[35329]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:24:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:42 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4222248055' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:24:43.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:42 smithi142 ceph-mon[35329]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:24:43.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:42 smithi050 ceph-mon[30378]: pgmap v302: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-12T03:24:43.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:42 smithi050 ceph-mon[30378]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:24:43.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:42 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4222248055' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:24:43.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:42 smithi050 ceph-mon[30378]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:24:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:42 smithi082 ceph-mon[35046]: pgmap v302: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-12T03:24:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:42 smithi082 ceph-mon[35046]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:24:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:42 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4222248055' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:24:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:42 smithi082 ceph-mon[35046]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 -- 172.21.15.50:0/4065869967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a410e3c0 msgr2=0x7fe3a4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/4065869967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a410e3c0 0x7fe3a4110810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009be0 tx=0x7fe39c02a750 comp rx=0 tx=0).stop 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 -- 172.21.15.50:0/4065869967 shutdown_connections 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/4065869967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a410e3c0 0x7fe3a4110810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/4065869967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4074f40 0x7fe3a410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/4065869967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3a4074620 0x7fe3a4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 -- 172.21.15.50:0/4065869967 >> 172.21.15.50:0/4065869967 conn(0x7fe3a40fd8c0 msgr2=0x7fe3a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:43.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 -- 172.21.15.50:0/4065869967 shutdown_connections 2024-09-12T03:24:43.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.299+0000 7fe3a9e00640 1 -- 172.21.15.50:0/4065869967 wait complete. 2024-09-12T03:24:43.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.300+0000 7fe3a9e00640 1 Processor -- start 2024-09-12T03:24:43.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 -- start start 2024-09-12T03:24:43.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 0x7fe3a419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:43.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3a4074f40 0x7fe3a419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:43.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a410e3c0 0x7fe3a419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:43.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41140f0 con 0x7fe3a410e3c0 2024-09-12T03:24:43.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4113f70 con 0x7fe3a4074620 2024-09-12T03:24:43.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a9e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4114270 con 0x7fe3a4074f40 2024-09-12T03:24:43.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 0x7fe3a419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:43.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 0x7fe3a419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:36502/0 (socket says 172.21.15.50:36502) 2024-09-12T03:24:43.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a410e3c0 0x7fe3a419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:43.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe3a2a00640 1 -- 172.21.15.50:0/2376159680 learned_addr learned my addr 172.21.15.50:0/2376159680 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:43.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.301+0000 7fe39a000640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3a4074f40 0x7fe3a419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a2a00640 1 -- 172.21.15.50:0/2376159680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3a4074f40 msgr2=0x7fe3a419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a2a00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3a4074f40 0x7fe3a419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a2a00640 1 -- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a410e3c0 msgr2=0x7fe3a419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a2a00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a410e3c0 0x7fe3a419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a2a00640 1 -- 172.21.15.50:0/2376159680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a419f640 con 0x7fe3a4074620 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a2a00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 0x7fe3a419a690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe39400ea60 tx=0x7fe39400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400cde0 con 0x7fe3a4074620 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3940035f0 con 0x7fe3a4074620 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a419f930 con 0x7fe3a4074620 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394010670 con 0x7fe3a4074620 2024-09-12T03:24:43.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.302+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41a7210 con 0x7fe3a4074620 2024-09-12T03:24:43.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.303+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a4074a00 con 0x7fe3a4074620 2024-09-12T03:24:43.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.305+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe3940037a0 con 0x7fe3a4074620 2024-09-12T03:24:43.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.307+0000 7fe3a8a00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe3880a2b80 0x7fe3880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:43.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.307+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fe3940cf790 con 0x7fe3a4074620 2024-09-12T03:24:43.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.307+0000 7fe39a000640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe3880a2b80 0x7fe3880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:43.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.308+0000 7fe39a000640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe3880a2b80 0x7fe3880a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0096c0 tx=0x7fe38c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:43.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.309+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3940541b0 con 0x7fe3a4074620 2024-09-12T03:24:43.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.531+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe3a419b9f0 con 0x7fe3a4074620 2024-09-12T03:24:43.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.705+0000 7fe3a8a00640 1 -- 172.21.15.50:0/2376159680 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fe39409c470 con 0x7fe3a4074620 2024-09-12T03:24:43.706 INFO:teuthology.orchestra.run.smithi050.stderr:added data pool 4 to fsmap 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe3880a2b80 msgr2=0x7fe3880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe3880a2b80 0x7fe3880a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0096c0 tx=0x7fe38c009260 comp rx=0 tx=0).stop 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 msgr2=0x7fe3a419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 0x7fe3a419a690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe39400ea60 tx=0x7fe39400ef20 comp rx=0 tx=0).stop 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 shutdown_connections 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe3880a2b80 0x7fe3880a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a410e3c0 0x7fe3a419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3a4074f40 0x7fe3a419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 --2- 172.21.15.50:0/2376159680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a4074620 0x7fe3a419a690 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:43.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 >> 172.21.15.50:0/2376159680 conn(0x7fe3a40fd8c0 msgr2=0x7fe3a40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:43.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 shutdown_connections 2024-09-12T03:24:43.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:43.708+0000 7fe3a9e00640 1 -- 172.21.15.50:0/2376159680 wait complete. 2024-09-12T03:24:44.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:43 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2376159680' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:24:44.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:43 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:24:44.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:43 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2376159680' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:24:44.057 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:43 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:24:44.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:43 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2376159680' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:24:44.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:43 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:24:44.510 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:44.896 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:44.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:44 smithi050 ceph-mon[30378]: pgmap v304: 129 pgs: 22 unknown, 107 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:44.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:44 smithi050 ceph-mon[30378]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:24:44.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:44 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:24:44.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:44 smithi050 ceph-mon[30378]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:44 smithi142 ceph-mon[35329]: pgmap v304: 129 pgs: 22 unknown, 107 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:44 smithi142 ceph-mon[35329]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:24:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:44 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:24:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:44 smithi142 ceph-mon[35329]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:44 smithi082 ceph-mon[35046]: pgmap v304: 129 pgs: 22 unknown, 107 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:44 smithi082 ceph-mon[35046]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:24:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:44 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:24:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:44 smithi082 ceph-mon[35046]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.488+0000 7fd2a0a00640 1 -- 172.21.15.50:0/1187325613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c104010 msgr2=0x7fd29c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.488+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/1187325613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c104010 0x7fd29c1043f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c02a410 comp rx=0 tx=0).stop 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 -- 172.21.15.50:0/1187325613 shutdown_connections 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/1187325613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c109140 0x7fd29c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/1187325613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29c1049c0 0x7fd29c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/1187325613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c104010 0x7fd29c1043f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 -- 172.21.15.50:0/1187325613 >> 172.21.15.50:0/1187325613 conn(0x7fd29c0ffa50 msgr2=0x7fd29c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 -- 172.21.15.50:0/1187325613 shutdown_connections 2024-09-12T03:24:45.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.489+0000 7fd2a0a00640 1 -- 172.21.15.50:0/1187325613 wait complete. 2024-09-12T03:24:45.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 Processor -- start 2024-09-12T03:24:45.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 -- start start 2024-09-12T03:24:45.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29c104010 0x7fd29c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c1049c0 0x7fd29c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 0x7fd29c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c114170 con 0x7fd29c109140 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c113ff0 con 0x7fd29c104010 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c1142f0 con 0x7fd29c1049c0 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.490+0000 7fd29a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 0x7fd29c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 0x7fd29c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46194/0 (socket says 172.21.15.50:46194) 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 -- 172.21.15.50:0/37258192 learned_addr learned my addr 172.21.15.50:0/37258192 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd299e00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29c104010 0x7fd29c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd291400640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c1049c0 0x7fd29c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 -- 172.21.15.50:0/37258192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c1049c0 msgr2=0x7fd29c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c1049c0 0x7fd29c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 -- 172.21.15.50:0/37258192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29c104010 msgr2=0x7fd29c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29c104010 0x7fd29c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 -- 172.21.15.50:0/37258192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c19f600 con 0x7fd29c109140 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.491+0000 7fd29a800640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 0x7fd29c19ef20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd29400eb10 tx=0x7fd29400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.492+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294004b20 con 0x7fd29c109140 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.492+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd294004cc0 con 0x7fd29c109140 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.492+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294010760 con 0x7fd29c109140 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.492+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29c19f8f0 con 0x7fd29c109140 2024-09-12T03:24:45.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.492+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29c1a71d0 con 0x7fd29c109140 2024-09-12T03:24:45.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.493+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29c1043f0 con 0x7fd29c109140 2024-09-12T03:24:45.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.495+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd294005020 con 0x7fd29c109140 2024-09-12T03:24:45.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.496+0000 7fd293400640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd2800a2c00 0x7fd2800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.496+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd2940cf890 con 0x7fd29c109140 2024-09-12T03:24:45.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.497+0000 7fd299e00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd2800a2c00 0x7fd2800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.497+0000 7fd299e00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd2800a2c00 0x7fd2800a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009230 tx=0x7fd28c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.501+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd29409c550 con 0x7fd29c109140 2024-09-12T03:24:45.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.725+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd29c19b9b0 con 0x7fd29c109140 2024-09-12T03:24:45.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.726+0000 7fd293400640 1 -- 172.21.15.50:0/37258192 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7fd29409be50 con 0x7fd29c109140 2024-09-12T03:24:45.727 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:45.727 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"btime":"2024-09-12T03:24:43:695300+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:24:43.534601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24536,"qdb_cluster":[24536]},"id":1}]} 2024-09-12T03:24:45.727 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd2800a2c00 msgr2=0x7fd2800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd2800a2c00 0x7fd2800a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009230 tx=0x7fd28c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 msgr2=0x7fd29c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 0x7fd29c19ef20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd29400eb10 tx=0x7fd29400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 shutdown_connections 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fd2800a2c00 0x7fd2800a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd29c109140 0x7fd29c19ef20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c1049c0 0x7fd29c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 --2- 172.21.15.50:0/37258192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29c104010 0x7fd29c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 >> 172.21.15.50:0/37258192 conn(0x7fd29c0ffa50 msgr2=0x7fd29c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 shutdown_connections 2024-09-12T03:24:45.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:45.728+0000 7fd2a0a00640 1 -- 172.21.15.50:0/37258192 wait complete. 2024-09-12T03:24:46.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:45 smithi142 ceph-mon[35329]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:46.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:45 smithi142 ceph-mon[35329]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:24:46.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:45 smithi142 ceph-mon[35329]: Cluster is now healthy 2024-09-12T03:24:46.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:45 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/37258192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:46.086 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:45 smithi050 ceph-mon[30378]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:46.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:45 smithi050 ceph-mon[30378]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:24:46.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:45 smithi050 ceph-mon[30378]: Cluster is now healthy 2024-09-12T03:24:46.087 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:45 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/37258192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:45 smithi082 ceph-mon[35046]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:45 smithi082 ceph-mon[35046]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:24:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:45 smithi082 ceph-mon[35046]: Cluster is now healthy 2024-09-12T03:24:46.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:45 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/37258192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:46.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:24:46.894 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.456+0000 7f0215a00640 1 -- 172.21.15.50:0/1378885810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210103ff0 msgr2=0x7f02101043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.456+0000 7f0215a00640 1 --2- 172.21.15.50:0/1378885810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210103ff0 0x7f02101043d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0200009900 tx=0x7f020002a410 comp rx=0 tx=0).stop 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 -- 172.21.15.50:0/1378885810 shutdown_connections 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 --2- 172.21.15.50:0/1378885810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210109120 0x7f0210110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 --2- 172.21.15.50:0/1378885810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 0x7f02101089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 --2- 172.21.15.50:0/1378885810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210103ff0 0x7f02101043d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 -- 172.21.15.50:0/1378885810 >> 172.21.15.50:0/1378885810 conn(0x7f02100ffa90 msgr2=0x7f0210101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:47.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 -- 172.21.15.50:0/1378885810 shutdown_connections 2024-09-12T03:24:47.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.457+0000 7f0215a00640 1 -- 172.21.15.50:0/1378885810 wait complete. 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.458+0000 7f0215a00640 1 Processor -- start 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.458+0000 7f0215a00640 1 -- start start 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.458+0000 7f0215a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210103ff0 0x7f021019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f0215a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 0x7f021019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f0215a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 0x7f021019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f0215a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0210114170 con 0x7f0210109120 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f0215a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0210113ff0 con 0x7f02101049a0 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f0215a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02101142f0 con 0x7f0210103ff0 2024-09-12T03:24:47.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f020f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 0x7f021019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:47.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f020ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210103ff0 0x7f021019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:47.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f020f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 0x7f021019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46206/0 (socket says 172.21.15.50:46206) 2024-09-12T03:24:47.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f020f400640 1 -- 172.21.15.50:0/957717835 learned_addr learned my addr 172.21.15.50:0/957717835 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:47.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f0206000640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 0x7f021019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:47.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f020f400640 1 -- 172.21.15.50:0/957717835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210103ff0 msgr2=0x7f021019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:47.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.459+0000 7f020f400640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210103ff0 0x7f021019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f020f400640 1 -- 172.21.15.50:0/957717835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 msgr2=0x7f021019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:47.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f020f400640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 0x7f021019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f020f400640 1 -- 172.21.15.50:0/957717835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021019f620 con 0x7f0210109120 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f0206000640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 0x7f021019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f020ea00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210103ff0 0x7f021019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f020f400640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 0x7f021019ef40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f020800eaa0 tx=0x7f020800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020800cdb0 con 0x7f0210109120 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021019f910 con 0x7f0210109120 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0208005020 con 0x7f0210109120 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080116b0 con 0x7f0210109120 2024-09-12T03:24:47.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.460+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101a71f0 con 0x7f0210109120 2024-09-12T03:24:47.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.462+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02101043d0 con 0x7f0210109120 2024-09-12T03:24:47.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.463+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0208005560 con 0x7f0210109120 2024-09-12T03:24:47.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.465+0000 7f0207e00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f40a2c00 0x7f01f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:47.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.465+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f02080cf3b0 con 0x7f0210109120 2024-09-12T03:24:47.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.465+0000 7f020ea00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f40a2c00 0x7f01f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:47.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.466+0000 7f020ea00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f40a2c00 0x7f01f40a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0200000c00 tx=0x7f0200031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:47.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.469+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0208016030 con 0x7f0210109120 2024-09-12T03:24:47.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.665+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f021010c300 con 0x7f0210109120 2024-09-12T03:24:47.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.667+0000 7f0207e00640 1 -- 172.21.15.50:0/957717835 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24154 (secure 0 0 0) 0x7f020809ba20 con 0x7f0210109120 2024-09-12T03:24:47.667 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:47.667 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":90,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:24:43.534465+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f40a2c00 msgr2=0x7f01f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f40a2c00 0x7f01f40a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0200000c00 tx=0x7f0200031510 comp rx=0 tx=0).stop 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 msgr2=0x7f021019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 0x7f021019ef40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f020800eaa0 tx=0x7f020800ef60 comp rx=0 tx=0).stop 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 shutdown_connections 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f01f40a2c00 0x7f01f40a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0210109120 0x7f021019ef40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101049a0 0x7f021019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 --2- 172.21.15.50:0/957717835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210103ff0 0x7f021019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:47.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 >> 172.21.15.50:0/957717835 conn(0x7f02100ffa90 msgr2=0x7f0210100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:47.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 shutdown_connections 2024-09-12T03:24:47.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:47.670+0000 7f0215a00640 1 -- 172.21.15.50:0/957717835 wait complete. 2024-09-12T03:24:48.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:47 smithi050 ceph-mon[30378]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:48.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:47 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/957717835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:47 smithi142 ceph-mon[35329]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:48.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:47 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/957717835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:47 smithi082 ceph-mon[35046]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:48.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:47 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/957717835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:48.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:24:48.865 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:49.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.432+0000 7f73d3a00640 1 -- 172.21.15.50:0/1668980962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc074590 msgr2=0x7f73cc074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.432+0000 7f73d3a00640 1 --2- 172.21.15.50:0/1668980962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc074590 0x7f73cc074970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009900 tx=0x7f73bc02a480 comp rx=0 tx=0).stop 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 -- 172.21.15.50:0/1668980962 shutdown_connections 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 --2- 172.21.15.50:0/1668980962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc112b90 0x7f73cc114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 --2- 172.21.15.50:0/1668980962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73cc074eb0 0x7f73cc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 --2- 172.21.15.50:0/1668980962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc074590 0x7f73cc074970 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 -- 172.21.15.50:0/1668980962 >> 172.21.15.50:0/1668980962 conn(0x7f73cc0fd8c0 msgr2=0x7f73cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 -- 172.21.15.50:0/1668980962 shutdown_connections 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.433+0000 7f73d3a00640 1 -- 172.21.15.50:0/1668980962 wait complete. 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 Processor -- start 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 -- start start 2024-09-12T03:24:49.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 0x7f73cc1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73cc074eb0 0x7f73cc1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc112b90 0x7f73cc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1185a0 con 0x7f73cc112b90 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc118420 con 0x7f73cc074eb0 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d3a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc118720 con 0x7f73cc074590 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 0x7f73cc1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 0x7f73cc1a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41544/0 (socket says 172.21.15.50:41544) 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc112b90 0x7f73cc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.434+0000 7f73d1000640 1 -- 172.21.15.50:0/2566901546 learned_addr learned my addr 172.21.15.50:0/2566901546 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73c3e00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73cc074eb0 0x7f73cc1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1000640 1 -- 172.21.15.50:0/2566901546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73cc074eb0 msgr2=0x7f73cc1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1000640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73cc074eb0 0x7f73cc1a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1000640 1 -- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc112b90 msgr2=0x7f73cc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:49.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1000640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc112b90 0x7f73cc19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1000640 1 -- 172.21.15.50:0/2566901546 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc19f000 con 0x7f73cc074590 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc112b90 0x7f73cc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.435+0000 7f73d1000640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 0x7f73cc1a4710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f73bc004770 tx=0x7f73bc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.436+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc03f070 con 0x7f73cc074590 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.436+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73bc031ca0 con 0x7f73cc074590 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.436+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc19f290 con 0x7f73cc074590 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.436+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc035030 con 0x7f73cc074590 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.436+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1ab5e0 con 0x7f73cc074590 2024-09-12T03:24:49.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.437+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc074970 con 0x7f73cc074590 2024-09-12T03:24:49.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.439+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73bc042070 con 0x7f73cc074590 2024-09-12T03:24:49.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.440+0000 7f73ca000640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73b00a2b80 0x7f73b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:49.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.440+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f73bc0ef920 con 0x7f73cc074590 2024-09-12T03:24:49.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.440+0000 7f73c3e00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73b00a2b80 0x7f73b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:49.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.441+0000 7f73c3e00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73b00a2b80 0x7f73b00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0ffec0 tx=0x7f73b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:49.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.444+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73bc0bbff0 con 0x7f73cc074590 2024-09-12T03:24:49.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.631+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73cc104fd0 con 0x7f73cc074590 2024-09-12T03:24:49.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.633+0000 7f73ca000640 1 -- 172.21.15.50:0/2566901546 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24154 (secure 0 0 0) 0x7f73bc0c02f0 con 0x7f73cc074590 2024-09-12T03:24:49.633 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:49.634 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":90,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:24:43.534465+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:24:49.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73b00a2b80 msgr2=0x7f73b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:49.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73b00a2b80 0x7f73b00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0ffec0 tx=0x7f73b4005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:49.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 msgr2=0x7f73cc1a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:49.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 0x7f73cc1a4710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f73bc004770 tx=0x7f73bc003c10 comp rx=0 tx=0).stop 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 shutdown_connections 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73b00a2b80 0x7f73b00a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc112b90 0x7f73cc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73cc074eb0 0x7f73cc1a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 --2- 172.21.15.50:0/2566901546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc074590 0x7f73cc1a4710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.636+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 >> 172.21.15.50:0/2566901546 conn(0x7f73cc0fd8c0 msgr2=0x7f73cc0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.637+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 shutdown_connections 2024-09-12T03:24:49.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:49.637+0000 7f73d3a00640 1 -- 172.21.15.50:0/2566901546 wait complete. 2024-09-12T03:24:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:49 smithi050 ceph-mon[30378]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:49.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:49 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2566901546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:50.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:49 smithi142 ceph-mon[35329]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:50.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:49 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2566901546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:50.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:49 smithi082 ceph-mon[35046]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:50.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:49 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2566901546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:50.514 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:50.894 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:51.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 -- 172.21.15.50:0/3780462396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528108f60 msgr2=0x7f0528110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:51.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 --2- 172.21.15.50:0/3780462396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528108f60 0x7f0528110ae0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0520009ca0 tx=0x7f052002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 -- 172.21.15.50:0/3780462396 shutdown_connections 2024-09-12T03:24:51.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 --2- 172.21.15.50:0/3780462396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528108f60 0x7f0528110ae0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 --2- 172.21.15.50:0/3780462396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f0528108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 --2- 172.21.15.50:0/3780462396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0528103e30 0x7f0528104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 -- 172.21.15.50:0/3780462396 >> 172.21.15.50:0/3780462396 conn(0x7f05280ff910 msgr2=0x7f0528101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 -- 172.21.15.50:0/3780462396 shutdown_connections 2024-09-12T03:24:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.423+0000 7f052d000640 1 -- 172.21.15.50:0/3780462396 wait complete. 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.424+0000 7f052d000640 1 Processor -- start 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.424+0000 7f052d000640 1 -- start start 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f052d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528103e30 0x7f052819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f052d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f052819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f052d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0528108f60 0x7f052819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f052d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0528113fc0 con 0x7f0528103e30 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f052d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0528113e40 con 0x7f0528108f60 2024-09-12T03:24:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f052d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0528114140 con 0x7f05281047e0 2024-09-12T03:24:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f0526a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f052819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f0527400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528103e30 0x7f052819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f0526a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f052819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41562/0 (socket says 172.21.15.50:41562) 2024-09-12T03:24:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f0526a00640 1 -- 172.21.15.50:0/1560020486 learned_addr learned my addr 172.21.15.50:0/1560020486 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.425+0000 7f0527e00640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0528108f60 0x7f052819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f0526a00640 1 -- 172.21.15.50:0/1560020486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0528108f60 msgr2=0x7f052819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f0526a00640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0528108f60 0x7f052819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f0526a00640 1 -- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528103e30 msgr2=0x7f052819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f0526a00640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528103e30 0x7f052819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f0526a00640 1 -- 172.21.15.50:0/1560020486 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819f3b0 con 0x7f05281047e0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f0526a00640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f052819a940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f051400e9f0 tx=0x7f051400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051400cd80 con 0x7f05281047e0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819f6a0 con 0x7f05281047e0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0514010910 con 0x7f05281047e0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05140188f0 con 0x7f05281047e0 2024-09-12T03:24:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.426+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a6f80 con 0x7f05281047e0 2024-09-12T03:24:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.427+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0528105310 con 0x7f05281047e0 2024-09-12T03:24:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.430+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0514010ac0 con 0x7f05281047e0 2024-09-12T03:24:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.431+0000 7f051fe00640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f050c0a2b80 0x7f050c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.431+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f05140cfb50 con 0x7f05281047e0 2024-09-12T03:24:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.431+0000 7f0527400640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f050c0a2b80 0x7f050c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.432+0000 7f0527400640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f050c0a2b80 0x7f050c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0518009bd0 tx=0x7f0518005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.433+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f051409c1c0 con 0x7f05281047e0 2024-09-12T03:24:51.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.655+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f052810bc10 con 0x7f05281047e0 2024-09-12T03:24:51.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.656+0000 7f051fe00640 1 -- 172.21.15.50:0/1560020486 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f05140a04c0 con 0x7f05281047e0 2024-09-12T03:24:51.656 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:51.656 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"btime":"2024-09-12T03:24:43:695300+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:24:43.534601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24536,"qdb_cluster":[24536]},"id":1}]} 2024-09-12T03:24:51.657 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2024-09-12T03:24:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f050c0a2b80 msgr2=0x7f050c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:51.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f050c0a2b80 0x7f050c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0518009bd0 tx=0x7f0518005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 msgr2=0x7f052819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f052819a940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f051400e9f0 tx=0x7f051400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 shutdown_connections 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f050c0a2b80 0x7f050c0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0528108f60 0x7f052819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05281047e0 0x7f052819a940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 --2- 172.21.15.50:0/1560020486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0528103e30 0x7f052819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 >> 172.21.15.50:0/1560020486 conn(0x7f05280ff910 msgr2=0x7f0528100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 shutdown_connections 2024-09-12T03:24:51.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:51.659+0000 7f052d000640 1 -- 172.21.15.50:0/1560020486 wait complete. 2024-09-12T03:24:51.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:51 smithi050 ceph-mon[30378]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:51.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:51 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1560020486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:51 smithi142 ceph-mon[35329]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:51 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1560020486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:52.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:51 smithi082 ceph-mon[35046]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:52.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:51 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1560020486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:52.338 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:24:52.716 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 -- 172.21.15.50:0/2389643672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 msgr2=0x7f58f4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 --2- 172.21.15.50:0/2389643672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f4076b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009900 tx=0x7f58e402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 -- 172.21.15.50:0/2389643672 shutdown_connections 2024-09-12T03:24:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 --2- 172.21.15.50:0/2389643672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58f4112870 0x7f58f4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 --2- 172.21.15.50:0/2389643672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f4077040 0x7f58f4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 --2- 172.21.15.50:0/2389643672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f4076b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 -- 172.21.15.50:0/2389643672 >> 172.21.15.50:0/2389643672 conn(0x7f58f40ff8d0 msgr2=0x7f58f4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:53.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 -- 172.21.15.50:0/2389643672 shutdown_connections 2024-09-12T03:24:53.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.233+0000 7f58f3e00640 1 -- 172.21.15.50:0/2389643672 wait complete. 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.234+0000 7f58f3e00640 1 Processor -- start 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.234+0000 7f58f3e00640 1 -- start start 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f4077040 0x7f58f41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58f4112870 0x7f58f419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41183e0 con 0x7f58f4076720 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4118260 con 0x7f58f4077040 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4118560 con 0x7f58f4112870 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f41a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38530/0 (socket says 172.21.15.50:38530) 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 -- 172.21.15.50:0/861621472 learned_addr learned my addr 172.21.15.50:0/861621472 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:53.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f3400640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58f4112870 0x7f58f419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:53.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58ea000640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f4077040 0x7f58f41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:53.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 -- 172.21.15.50:0/861621472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58f4112870 msgr2=0x7f58f419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58f4112870 0x7f58f419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 -- 172.21.15.50:0/861621472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f4077040 msgr2=0x7f58f41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f4077040 0x7f58f41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.235+0000 7f58f2a00640 1 -- 172.21.15.50:0/861621472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f419efe0 con 0x7f58f4076720 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.236+0000 7f58f2a00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f41a46c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006010 tx=0x7f58e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.236+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e403f030 con 0x7f58f4076720 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.236+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e4003ce0 con 0x7f58f4076720 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.236+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4035070 con 0x7f58f4076720 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.236+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f419f270 con 0x7f58f4076720 2024-09-12T03:24:53.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.236+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f41ab520 con 0x7f58f4076720 2024-09-12T03:24:53.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.238+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f4077e60 con 0x7f58f4076720 2024-09-12T03:24:53.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.239+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f58e4042030 con 0x7f58f4076720 2024-09-12T03:24:53.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.241+0000 7f58ebe00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58d80a2c00 0x7f58d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:53.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.241+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f58e40ef790 con 0x7f58f4076720 2024-09-12T03:24:53.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.241+0000 7f58ea000640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58d80a2c00 0x7f58d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:53.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.242+0000 7f58ea000640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58d80a2c00 0x7f58d80a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58f4079320 tx=0x7f58dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:53.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.245+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58e40bbec0 con 0x7f58f4076720 2024-09-12T03:24:53.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.433+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58f410da00 con 0x7f58f4076720 2024-09-12T03:24:53.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.435+0000 7f58ebe00640 1 -- 172.21.15.50:0/861621472 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24154 (secure 0 0 0) 0x7f58e40c01c0 con 0x7f58f4076720 2024-09-12T03:24:53.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:24:53.435 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":90,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:24:43.534465+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:24:53.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.437+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58d80a2c00 msgr2=0x7f58d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:53.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.437+0000 7f58f3e00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58d80a2c00 0x7f58d80a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58f4079320 tx=0x7f58dc005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.437+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 msgr2=0x7f58f41a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.437+0000 7f58f3e00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f41a46c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006010 tx=0x7f58e40042f0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 shutdown_connections 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58d80a2c00 0x7f58d80a50c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58f4112870 0x7f58f419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f4077040 0x7f58f41a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 --2- 172.21.15.50:0/861621472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58f4076720 0x7f58f41a46c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 >> 172.21.15.50:0/861621472 conn(0x7f58f40ff8d0 msgr2=0x7f58f4101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 shutdown_connections 2024-09-12T03:24:53.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:53.438+0000 7f58f3e00640 1 -- 172.21.15.50:0/861621472 wait complete. 2024-09-12T03:24:54.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:53 smithi142 ceph-mon[35329]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:54.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:53 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/861621472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:54.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:53 smithi082 ceph-mon[35046]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:54.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:53 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/861621472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:54.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:53 smithi050 ceph-mon[30378]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:54.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:53 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/861621472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:24:54.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:24:32.635781+0000', 'modified': '2024-09-12T03:24:43.534601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24536}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.82:6839/2348373979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2348373979}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2348373979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24536, 'qdb_cluster': [24536]} 2024-09-12T03:24:54.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:24:54.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:24:54.645 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 -- 172.21.15.50:0/64469531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c104010 msgr2=0x7ff82c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 --2- 172.21.15.50:0/64469531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c104010 0x7ff82c1043f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009900 tx=0x7ff81c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 -- 172.21.15.50:0/64469531 shutdown_connections 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 --2- 172.21.15.50:0/64469531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 0x7ff82c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 --2- 172.21.15.50:0/64469531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c1049c0 0x7ff82c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 --2- 172.21.15.50:0/64469531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c104010 0x7ff82c1043f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 -- 172.21.15.50:0/64469531 >> 172.21.15.50:0/64469531 conn(0x7ff82c0ffa50 msgr2=0x7ff82c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 -- 172.21.15.50:0/64469531 shutdown_connections 2024-09-12T03:24:55.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.153+0000 7ff833000640 1 -- 172.21.15.50:0/64469531 wait complete. 2024-09-12T03:24:55.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 Processor -- start 2024-09-12T03:24:55.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 -- start start 2024-09-12T03:24:55.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 0x7ff82c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c1049c0 0x7ff82c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 0x7ff82c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c1141b0 con 0x7ff82c1049c0 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c114030 con 0x7ff82c109140 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.154+0000 7ff833000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c114330 con 0x7ff82c104010 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 0x7ff82c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff823400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c1049c0 0x7ff82c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 0x7ff82c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41608/0 (socket says 172.21.15.50:41608) 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 -- 172.21.15.50:0/4219786197 learned_addr learned my addr 172.21.15.50:0/4219786197 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:55.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff831000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 0x7ff82c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 -- 172.21.15.50:0/4219786197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 msgr2=0x7ff82c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 0x7ff82c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 -- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c1049c0 msgr2=0x7ff82c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c1049c0 0x7ff82c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 -- 172.21.15.50:0/4219786197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c19f600 con 0x7ff82c104010 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff831000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 0x7ff82c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.155+0000 7ff82be00640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 0x7ff82c19a650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff81c004770 tx=0x7ff81c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.156+0000 7ff829600640 1 -- 172.21.15.50:0/4219786197 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c03f070 con 0x7ff82c104010 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.156+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c19f890 con 0x7ff82c104010 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.156+0000 7ff829600640 1 -- 172.21.15.50:0/4219786197 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c031ca0 con 0x7ff82c104010 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.156+0000 7ff829600640 1 -- 172.21.15.50:0/4219786197 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c035030 con 0x7ff82c104010 2024-09-12T03:24:55.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.156+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c1a71d0 con 0x7ff82c104010 2024-09-12T03:24:55.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 Processor -- start 2024-09-12T03:24:55.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 -- start start 2024-09-12T03:24:55.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff829600640 1 -- 172.21.15.50:0/4219786197 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff81c042070 con 0x7ff82c104010 2024-09-12T03:24:55.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672988fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d67298b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d6728e40b0 con 0x55d67297ec50 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d67288bab0 con 0x55d672989510 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.159+0000 7ff835ab4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d6728e0070 con 0x55d67297e330 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41612/0 (socket says 172.21.15.50:41612) 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 -- 172.21.15.50:0/1832290326 learned_addr learned my addr 172.21.15.50:0/1832290326 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff82be00640 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672988fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff823400640 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d67298b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 -- 172.21.15.50:0/1832290326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 msgr2=0x55d67298b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d67298b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 -- 172.21.15.50:0/1832290326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 msgr2=0x55d672988fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672988fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 -- 172.21.15.50:0/1832290326 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6728984a0 con 0x55d67297e330 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.160+0000 7ff831000640 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297e710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff824010710 tx=0x7ff824010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd98aa148d63a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.161+0000 7ff821600640 1 -- 172.21.15.50:0/1832290326 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82400e8d0 con 0x55d67297e330 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.161+0000 7ff821600640 1 -- 172.21.15.50:0/1832290326 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff824012450 con 0x55d67297e330 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.161+0000 7ff821600640 1 -- 172.21.15.50:0/1832290326 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82401c650 con 0x55d67297e330 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.161+0000 7ff829600640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8100a2b80 0x7ff8100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.161+0000 7ff829600640 1 -- 172.21.15.50:0/4219786197 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff81c0ef5f0 con 0x7ff82c104010 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.161+0000 7ff82be00640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8100a2b80 0x7ff8100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 -- 172.21.15.50:0/1832290326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 msgr2=0x55d67297e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297e710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff824010710 tx=0x7ff824010bd0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 -- 172.21.15.50:0/1832290326 shutdown_connections 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d67298b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672988fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1832290326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297e710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 -- 172.21.15.50:0/1832290326 >> 172.21.15.50:0/1832290326 conn(0x55d67297a8c0 msgr2=0x55d67297aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 -- 172.21.15.50:0/1832290326 shutdown_connections 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff82be00640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8100a2b80 0x7ff8100a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff81c031550 tx=0x7ff81c0f50e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:55.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 -- 172.21.15.50:0/1832290326 wait complete. 2024-09-12T03:24:55.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.162+0000 7ff835ab4740 1 -- start start 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 Processor -- start 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d6729865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672987070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d67298c060 con 0x55d67297ec50 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d67286c920 con 0x55d672989510 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d67298c1e0 con 0x55d67297e330 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff835ab4740 10 client.0 _ll_register_callbacks cb 0x55d6728908a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.163+0000 7ff823400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38564/0 (socket says 172.21.15.50:38564) 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 -- 172.21.15.50:0/1713676139 learned_addr learned my addr 172.21.15.50:0/1713676139 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff831000640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672987070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff831000640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d6729865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 -- 172.21.15.50:0/1713676139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 msgr2=0x55d6729865f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d6729865f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 -- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 msgr2=0x55d672987070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672987070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 -- 172.21.15.50:0/1713676139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6726552b0 con 0x55d67297ec50 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff831000640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672987070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff823400640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8140126e0 tx=0x7ff814012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eba759c0c15d862 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.164+0000 7ff81a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff81a000640 1 -- 172.21.15.50:0/1713676139 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400f920 con 0x55d67297ec50 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff824012450 con 0x55d67297ec50 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff81a000640 1 -- 172.21.15.50:0/1713676139 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff81400fac0 con 0x55d67297ec50 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff81a000640 1 -- 172.21.15.50:0/1713676139 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff81400fcc0 con 0x55d67297ec50 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff835ab4740 10 client.14610 fetch_fsmap learned FSMap version 17 2024-09-12T03:24:55.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55d672997f30 con 0x55d67297ec50 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff81a000640 1 -- 172.21.15.50:0/1713676139 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff824012450 con 0x55d67297ec50 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.165+0000 7ff81a000640 1 -- 172.21.15.50:0/1713676139 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7ff81401b3c0 con 0x55d67297ec50 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff835ab4740 10 client.14610 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff835ab4740 10 client.14610 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff835ab4740 20 client.14610 populate_metadata read hostname 'smithi050' 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d67299a570 0x55d67299b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff835ab4740 4 client.14610 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6728f1620 con 0x55d67299a570 2024-09-12T03:24:55.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.166+0000 7ff831000640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d67299a570 0x55d67299b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.167+0000 7ff831000640 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d67299a570 0x55d67299b9e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c9606e848f23dc server_cookie=742c7aa4579f6076 in_seq=0 out_seq=0 2024-09-12T03:24:55.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.167+0000 7ff81a000640 10 client.14610 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:24:55.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff81a000640 1 -- 172.21.15.50:0/1713676139 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6728f1620 con 0x55d67299a570 2024-09-12T03:24:55.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff81a000640 10 client.14610 handle_command_reply: tid=0 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 client.14610 shutdown 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 msgr2=0x55d672986b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8140126e0 tx=0x7ff814012ba0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 shutdown_connections 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672987070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d67299a570 0x55d67299b9e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1713676139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d6729865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 >> 172.21.15.50:0/1713676139 conn(0x55d67297a8c0 msgr2=0x55d6729805f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff81a000640 0 client.14610 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 shutdown_connections 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 1 -- 172.21.15.50:0/1713676139 wait complete. 2024-09-12T03:24:55.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.168+0000 7ff835ab4740 20 client.14610 trim_cache size 0 max 16384 2024-09-12T03:24:55.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 Processor -- start 2024-09-12T03:24:55.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 -- start start 2024-09-12T03:24:55.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672983890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d672999d20 con 0x55d67297ec50 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d67299a3f0 con 0x55d672989510 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff835ab4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d67299aac0 con 0x55d67297e330 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff823400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff823400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38574/0 (socket says 172.21.15.50:38574) 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.187+0000 7ff823400640 1 -- 172.21.15.50:0/1317274234 learned_addr learned my addr 172.21.15.50:0/1317274234 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff831000640 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff831000640 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672983890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff823400640 1 -- 172.21.15.50:0/1317274234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 msgr2=0x55d67297aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff823400640 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff823400640 1 -- 172.21.15.50:0/1317274234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 msgr2=0x55d672983890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff823400640 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672983890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff823400640 1 -- 172.21.15.50:0/1317274234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67299af70 con 0x55d67297ec50 2024-09-12T03:24:55.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff831000640 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672983890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff823400640 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986ec0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff81401b8b0 tx=0x7ff8140096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f390b5d37757dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.188+0000 7ff820c00640 1 -- 172.21.15.50:0/1317274234 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400f920 con 0x55d67297ec50 2024-09-12T03:24:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff820c00640 1 -- 172.21.15.50:0/1317274234 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81400fac0 con 0x55d67297ec50 2024-09-12T03:24:55.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff820c00640 1 -- 172.21.15.50:0/1317274234 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814004640 con 0x55d67297ec50 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 -- 172.21.15.50:0/1317274234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 msgr2=0x55d672986ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986ec0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff81401b8b0 tx=0x7ff8140096a0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 -- 172.21.15.50:0/1317274234 shutdown_connections 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d672989510 0x55d672983890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d67297ec50 0x55d672986ec0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1317274234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297e330 0x55d67297aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 -- 172.21.15.50:0/1317274234 >> 172.21.15.50:0/1317274234 conn(0x55d67297a8c0 msgr2=0x55d672984ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 -- 172.21.15.50:0/1317274234 shutdown_connections 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.189+0000 7ff835ab4740 1 -- 172.21.15.50:0/1317274234 wait complete. 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 -- start start 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 Processor -- start 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d67297e330 0x55d6729a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297ec50 0x55d6729a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 0x55d67299ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d67299b860 con 0x55d672989510 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d67299b190 con 0x55d67297e330 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d67299bf30 con 0x55d67297ec50 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.190+0000 7ff835ab4740 10 client.0 _ll_register_callbacks cb 0x7ff814009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d67297e330 0x55d6729a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 0x55d67299ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 0x55d67299ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38580/0 (socket says 172.21.15.50:38580) 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 -- 172.21.15.50:0/1528038426 learned_addr learned my addr 172.21.15.50:0/1528038426 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:55.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff823400640 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297ec50 0x55d6729a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 -- 172.21.15.50:0/1528038426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297ec50 msgr2=0x55d6729a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297ec50 0x55d6729a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 -- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d67297e330 msgr2=0x55d6729a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d67297e330 0x55d6729a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 -- 172.21.15.50:0/1528038426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6729a48f0 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff831000640 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 0x55d67299ebf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff824012f00 tx=0x7ff824012df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71406c2eb432a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.191+0000 7ff81a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff81a000640 1 -- 172.21.15.50:0/1528038426 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824026040 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81400fac0 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff81a000640 1 -- 172.21.15.50:0/1528038426 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff82400e870 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff81a000640 1 -- 172.21.15.50:0/1528038426 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff82403e050 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff835ab4740 10 client.14616 fetch_fsmap learned FSMap version 17 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55d672a0daa0 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff81a000640 1 -- 172.21.15.50:0/1528038426 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81400fac0 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff81a000640 1 -- 172.21.15.50:0/1528038426 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7ff82402c040 con 0x55d672989510 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.192+0000 7ff835ab4740 10 client.14616 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff835ab4740 10 client.14616 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:24:55.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff835ab4740 20 client.14616 populate_metadata read hostname 'smithi050' 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d672a122c0 0x55d672a10fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff835ab4740 4 client.14616 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "status"}) -- 0x55d6728ebb30 con 0x55d672a122c0 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff823400640 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d672a122c0 0x55d672a10fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff823400640 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d672a122c0 0x55d672a10fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c473c6f08847ed05 server_cookie=de1f6a84c89a4dc7 in_seq=0 out_seq=0 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.193+0000 7ff81a000640 10 client.14616 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff81a000640 1 -- 172.21.15.50:0/1528038426 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d6728ebb30 con 0x55d672a122c0 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff81a000640 10 client.14616 handle_command_reply: tid=0 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 client.14616 shutdown 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 msgr2=0x55d67299ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 0x55d67299ebf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff824012f00 tx=0x7ff824012df0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 shutdown_connections 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d672989510 0x55d67299ebf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55d672a122c0 0x55d672a10fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d67297ec50 0x55d6729a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 --2- 172.21.15.50:0/1528038426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d67297e330 0x55d6729a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 >> 172.21.15.50:0/1528038426 conn(0x55d67297a8c0 msgr2=0x55d67297fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff81a000640 0 client.14616 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 shutdown_connections 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 1 -- 172.21.15.50:0/1528038426 wait complete. 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.194+0000 7ff835ab4740 20 client.14616 trim_cache size 0 max 16384 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-12T03:24:55.197 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24536, 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 21.556237550999999, 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 17, 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 90, 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 90, 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 35.146736033000003 2024-09-12T03:24:55.198 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8100a2b80 msgr2=0x7ff8100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8100a2b80 0x7ff8100a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff81c031550 tx=0x7ff81c0f50e0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 msgr2=0x7ff82c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 0x7ff82c19a650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff81c004770 tx=0x7ff81c003c10 comp rx=0 tx=0).stop 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 shutdown_connections 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8100a2b80 0x7ff8100a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c109140 0x7ff82c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff82c1049c0 0x7ff82c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 --2- 172.21.15.50:0/4219786197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82c104010 0x7ff82c19a650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 >> 172.21.15.50:0/4219786197 conn(0x7ff82c0ffa50 msgr2=0x7ff82c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 shutdown_connections 2024-09-12T03:24:55.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:55.195+0000 7ff833000640 1 -- 172.21.15.50:0/4219786197 wait complete. 2024-09-12T03:24:55.911 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:55 smithi050 ceph-mon[30378]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:56.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:55 smithi142 ceph-mon[35329]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:56.069 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:24:56.069 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:24:56.069 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:24:56.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:55 smithi082 ceph-mon[35046]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:56.445 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 -- 172.21.15.50:0/1083768355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc18103e30 msgr2=0x7fcc18104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 --2- 172.21.15.50:0/1083768355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc18103e30 0x7fcc18104210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009900 tx=0x7fcc0802a410 comp rx=0 tx=0).stop 2024-09-12T03:24:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 -- 172.21.15.50:0/1083768355 shutdown_connections 2024-09-12T03:24:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 --2- 172.21.15.50:0/1083768355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18108f60 0x7fcc18110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 --2- 172.21.15.50:0/1083768355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc181047e0 0x7fcc18108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 --2- 172.21.15.50:0/1083768355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc18103e30 0x7fcc18104210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 -- 172.21.15.50:0/1083768355 >> 172.21.15.50:0/1083768355 conn(0x7fcc180ff910 msgr2=0x7fcc18101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 -- 172.21.15.50:0/1083768355 shutdown_connections 2024-09-12T03:24:57.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.011+0000 7fcc17200640 1 -- 172.21.15.50:0/1083768355 wait complete. 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 Processor -- start 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 -- start start 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 0x7fcc1819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc181047e0 0x7fcc1819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc18108f60 0x7fcc1819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18114010 con 0x7fcc181047e0 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18113e90 con 0x7fcc18108f60 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.012+0000 7fcc17200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18114190 con 0x7fcc18103e30 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc0fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc181047e0 0x7fcc1819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 0x7fcc1819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 0x7fcc1819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41658/0 (socket says 172.21.15.50:41658) 2024-09-12T03:24:57.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 -- 172.21.15.50:0/1568685947 learned_addr learned my addr 172.21.15.50:0/1568685947 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc16800640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc18108f60 0x7fcc1819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 -- 172.21.15.50:0/1568685947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc18108f60 msgr2=0x7fcc1819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc18108f60 0x7fcc1819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 -- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc181047e0 msgr2=0x7fcc1819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc181047e0 0x7fcc1819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.013+0000 7fcc15e00640 1 -- 172.21.15.50:0/1568685947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1819f480 con 0x7fcc18103e30 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.014+0000 7fcc15e00640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 0x7fcc1819a4d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc08004770 tx=0x7fcc08003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.014+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0803f070 con 0x7fcc18103e30 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.014+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc08031ca0 con 0x7fcc18103e30 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.014+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1819f710 con 0x7fcc18103e30 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.014+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08035030 con 0x7fcc18103e30 2024-09-12T03:24:57.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.014+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc181a7050 con 0x7fcc18103e30 2024-09-12T03:24:57.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.015+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc18105310 con 0x7fcc18103e30 2024-09-12T03:24:57.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.017+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc08042070 con 0x7fcc18103e30 2024-09-12T03:24:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.018+0000 7fcc0ea00640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.019+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcc0806de00 con 0x7fcc18103e30 2024-09-12T03:24:57.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.019+0000 7fcc0fe00640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:57.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.019+0000 7fcc0fe00640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc000095f0 tx=0x7fcc00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:57.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.021+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc080bc090 con 0x7fcc18103e30 2024-09-12T03:24:57.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.246+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcc1810c140 con 0x7fcc18103e30 2024-09-12T03:24:57.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.247+0000 7fcc0ea00640 1 -- 172.21.15.50:0/1568685947 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fcc080c0390 con 0x7fcc18103e30 2024-09-12T03:24:57.247 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T03:24:57.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.249+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbfc0a2b80 msgr2=0x7fcbfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.249+0000 7fcc17200640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc000095f0 tx=0x7fcc00005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:57.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.249+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 msgr2=0x7fcc1819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.249+0000 7fcc17200640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 0x7fcc1819a4d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc08004770 tx=0x7fcc08003c10 comp rx=0 tx=0).stop 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 shutdown_connections 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbfc0a2b80 0x7fcbfc0a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc18108f60 0x7fcc1819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc181047e0 0x7fcc1819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 --2- 172.21.15.50:0/1568685947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc18103e30 0x7fcc1819a4d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 >> 172.21.15.50:0/1568685947 conn(0x7fcc180ff910 msgr2=0x7fcc18100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 shutdown_connections 2024-09-12T03:24:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:57.250+0000 7fcc17200640 1 -- 172.21.15.50:0/1568685947 wait complete. 2024-09-12T03:24:57.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:57 smithi050 ceph-mon[30378]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:57.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:57 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1568685947' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:24:57.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:24:57.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:24:57.977 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T03:24:57.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:57.977 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-12T03:24:58.013 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:58.014 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42210sec preferred_lft 42210sec 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:58.015 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-12T03:24:58.016 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:24:58.016 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:58.016 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:24:58.016 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:24:58.016 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-09-12T03:24:58.016 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-09-12T03:24:58.017 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:24:58.017 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:24:58.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:57 smithi142 ceph-mon[35329]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:58.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:57 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1568685947' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:24:58.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:57 smithi082 ceph-mon[35046]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:24:58.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:57 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1568685947' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:24:58.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:58.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:58.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:58.433 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:24:58.490 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-12T03:24:58.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:58.493 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-12T03:24:58.547 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-12T03:24:58.548 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:24:58.548 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:24:58.548 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:24:58.548 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:24:58.548 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:58.548 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:24:58.548 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:24:58.549 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:24:58.549 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:24:58.549 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:24:58.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:59.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:59.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:59.168 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-12T03:24:59.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:59.204 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-09-12T03:24:59.280 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:59.280 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:24:59.280 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:24:59.280 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-12T03:24:59.280 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:24:59.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:59.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:59.459 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-12T03:24:59.459 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:24:59.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:59.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:59.705 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:59.705 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:24:59.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-12T03:24:59.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:24:59.705 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:24:59.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:24:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:59.874 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-12T03:24:59.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:24:59.874 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:24:59.913 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T03:24:59.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:24:59.914 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:25:00.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-09-12T03:25:00.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:24:59 smithi142 ceph-mon[35329]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:00.097 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:24:59 smithi050 ceph-mon[30378]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:00.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:24:59 smithi082 ceph-mon[35046]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:00.110 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:25:00.188 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-09-12T03:25:00.189 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:25:00.190 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/9af5eb28942792c7b68270920ed7fa4cfa7522d0705d3aaabbd28a334eaf0661/merged 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/9b4eaee076d5571eb8b557c839c858efb75f090ec90bb5beee701f2a8c8f35cb/merged 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:25:00.191 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f4797ff6d83e223402f62620dc7c2b4f2743a0f85bc0366e3b743075927f4f1e/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/0c83cbf5a4223f829b7dbb76843ac699655830b22d57af380c64409901fd0cc6/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed625c35a6bcc4056c3a6499276ee5143c3c293f5cd1c68ee3c73fbcf0edb9c1/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7810e6444dc67f1a8033c79274f9897d5de0be615b55c5b7dcec4dee4ebab25d/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/a3b3d26d1993e6261fcca06baae99dde6f3cc03496e0066acb1374394df5d7d2/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/bd50b2e009cfceff384aeea5fe3d21dca86e726f95ad08aef129fce32f1cd914/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/2a4ffe293c0771df1d656b9ba5e5d4c3cc1ac6f3d1f660850b5d54f5ef0ad1b4/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/43de6d5c3477ff70f77711caedf3f7f5d8565c26c7dc04a57344892fba3751e1/merged 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-09-12T03:25:00.192 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:25:00.193 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:25:00.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:00.194 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-09-12T03:25:00.270 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T03:25:00.270 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-12T03:25:00.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-09-12T03:25:00.371 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:25:00.408 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-09-12T03:25:00.408 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-09-12T03:25:00.408 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-09-12T03:25:00.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/9af5eb28942792c7b68270920ed7fa4cfa7522d0705d3aaabbd28a334eaf0661/merged 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/9b4eaee076d5571eb8b557c839c858efb75f090ec90bb5beee701f2a8c8f35cb/merged 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-09-12T03:25:00.410 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f4797ff6d83e223402f62620dc7c2b4f2743a0f85bc0366e3b743075927f4f1e/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/0c83cbf5a4223f829b7dbb76843ac699655830b22d57af380c64409901fd0cc6/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed625c35a6bcc4056c3a6499276ee5143c3c293f5cd1c68ee3c73fbcf0edb9c1/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7810e6444dc67f1a8033c79274f9897d5de0be615b55c5b7dcec4dee4ebab25d/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/a3b3d26d1993e6261fcca06baae99dde6f3cc03496e0066acb1374394df5d7d2/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/bd50b2e009cfceff384aeea5fe3d21dca86e726f95ad08aef129fce32f1cd914/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/2a4ffe293c0771df1d656b9ba5e5d4c3cc1ac6f3d1f660850b5d54f5ef0ad1b4/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/43de6d5c3477ff70f77711caedf3f7f5d8565c26c7dc04a57344892fba3751e1/merged 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:25:00.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:25:00.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:00.412 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-09-12T03:25:00.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:2024-09-12T03:25:00.503+0000 7f7116b58480 -1 init, newargv = 0x55cb8beed400 newargc=15 2024-09-12T03:25:00.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[74734]: starting ceph client 2024-09-12T03:25:00.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[74734]: starting fuse 2024-09-12T03:25:01.592 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-09-12T03:25:01.657 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:25:01.696 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:25:01.697 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/9af5eb28942792c7b68270920ed7fa4cfa7522d0705d3aaabbd28a334eaf0661/merged 2024-09-12T03:25:01.698 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/9b4eaee076d5571eb8b557c839c858efb75f090ec90bb5beee701f2a8c8f35cb/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f4797ff6d83e223402f62620dc7c2b4f2743a0f85bc0366e3b743075927f4f1e/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/0c83cbf5a4223f829b7dbb76843ac699655830b22d57af380c64409901fd0cc6/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed625c35a6bcc4056c3a6499276ee5143c3c293f5cd1c68ee3c73fbcf0edb9c1/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7810e6444dc67f1a8033c79274f9897d5de0be615b55c5b7dcec4dee4ebab25d/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/a3b3d26d1993e6261fcca06baae99dde6f3cc03496e0066acb1374394df5d7d2/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/bd50b2e009cfceff384aeea5fe3d21dca86e726f95ad08aef129fce32f1cd914/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/2a4ffe293c0771df1d656b9ba5e5d4c3cc1ac6f3d1f660850b5d54f5ef0ad1b4/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/43de6d5c3477ff70f77711caedf3f7f5d8565c26c7dc04a57344892fba3751e1/merged 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:25:01.699 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:25:01.700 INFO:teuthology.orchestra.run.smithi050.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-12T03:25:01.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:01.701 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-09-12T03:25:01.778 INFO:teuthology.orchestra.run.smithi050.stdout:120 2024-09-12T03:25:01.778 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-12T03:25:01.778 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2024-09-12T03:25:01.778 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> import re 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> import os 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> 2024-09-12T03:25:01.779 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> return f 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.admin")) 2024-09-12T03:25:01.780 DEBUG:teuthology.orchestra.run.smithi050:> ' 2024-09-12T03:25:01.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:01 smithi050 ceph-mon[30378]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:01.944 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.admin.74734.asok 2024-09-12T03:25:01.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:25:01.958 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74734.asok 2024-09-12T03:25:01.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:01.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74734.asok status 2024-09-12T03:25:02.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:01 smithi142 ceph-mon[35329]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:02.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:01 smithi082 ceph-mon[35046]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "admin", 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "74734", 2024-09-12T03:25:02.145 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 14628, 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 14628 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-09-12T03:25:02.146 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2038318345 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2038318345 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.14628 192.168.0.1:0/2038318345", 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/2038318345", 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 17, 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 90, 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2024-09-12T03:25:02.147 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2024-09-12T03:25:02.148 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2024-09-12T03:25:02.148 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:25:02.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:02.157 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2024-09-12T03:25:02.158 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:25:02.194 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2024-09-12T03:25:02.194 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:25:02.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:02.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:25:02.294 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-12T03:25:02.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:25:02.403 INFO:teuthology.orchestra.run.smithi050.stdout:# file: . 2024-09-12T03:25:02.403 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-12T03:25:02.403 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:02.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:25:02.411 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:25:02.411 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:25:02.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:25:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 -- 172.21.15.50:0/3953605462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0103e50 msgr2=0x7f5ce0104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/3953605462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0103e50 0x7f5ce0104230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009860 tx=0x7f5cd002a340 comp rx=0 tx=0).stop 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 -- 172.21.15.50:0/3953605462 shutdown_connections 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/3953605462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ce0109090 0x7f5ce0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/3953605462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0104800 0x7f5ce0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/3953605462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0103e50 0x7f5ce0104230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 -- 172.21.15.50:0/3953605462 >> 172.21.15.50:0/3953605462 conn(0x7f5ce00ff8b0 msgr2=0x7f5ce0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 -- 172.21.15.50:0/3953605462 shutdown_connections 2024-09-12T03:25:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.681+0000 7f5ce4e00640 1 -- 172.21.15.50:0/3953605462 wait complete. 2024-09-12T03:25:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.682+0000 7f5ce4e00640 1 Processor -- start 2024-09-12T03:25:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.682+0000 7f5ce4e00640 1 -- start start 2024-09-12T03:25:03.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5ce4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 0x7f5ce019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:03.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5ce4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ce0104800 0x7f5ce019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5ce4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0109090 0x7f5ce019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5ce4e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01140b0 con 0x7f5ce0103e50 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5ce4e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0113f30 con 0x7f5ce0109090 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5ce4e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0114230 con 0x7f5ce0104800 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 0x7f5ce019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 0x7f5ce019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56522/0 (socket says 172.21.15.50:56522) 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 -- 172.21.15.50:0/838592314 learned_addr learned my addr 172.21.15.50:0/838592314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdfe00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0109090 0x7f5ce019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cd7e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ce0104800 0x7f5ce019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 -- 172.21.15.50:0/838592314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ce0104800 msgr2=0x7f5ce019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ce0104800 0x7f5ce019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 -- 172.21.15.50:0/838592314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0109090 msgr2=0x7f5ce019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0109090 0x7f5ce019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.683+0000 7f5cdf400640 1 -- 172.21.15.50:0/838592314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce019f520 con 0x7f5ce0103e50 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.684+0000 7f5cdf400640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 0x7f5ce019a570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0035040 tx=0x7f5cd0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.684+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd003f020 con 0x7f5ce0103e50 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.684+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce019f7b0 con 0x7f5ce0103e50 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.684+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0032770 con 0x7f5ce0103e50 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.684+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0037040 con 0x7f5ce0103e50 2024-09-12T03:25:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.684+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a70f0 con 0x7f5ce0103e50 2024-09-12T03:25:03.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.686+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0105330 con 0x7f5ce0103e50 2024-09-12T03:25:03.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.688+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5cd0042020 con 0x7f5ce0103e50 2024-09-12T03:25:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.689+0000 7f5cdcc00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cc40a2b80 0x7f5cc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.689+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5cd00ef4a0 con 0x7f5ce0103e50 2024-09-12T03:25:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.689+0000 7f5cd7e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cc40a2b80 0x7f5cc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.690+0000 7f5cd7e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cc40a2b80 0x7f5cc40a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00ffdd0 tx=0x7f5cc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:03.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.694+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cd0049020 con 0x7f5ce0103e50 2024-09-12T03:25:03.894 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:03 smithi050 ceph-mon[30378]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.893+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ce019b900 con 0x7f5ce0103e50 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.894+0000 7f5cdcc00640 1 -- 172.21.15.50:0/838592314 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+592 (secure 0 0 0) 0x7f5ce019b900 con 0x7f5ce0103e50 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3330091767 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2850856182 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1363145359 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3574998620 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2127938282 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/1939431272 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2850856182 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:03.896 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/1939431272 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:03.896 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1226501005 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:03.896 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1765792264 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:03.896 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-09-12T03:25:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.896+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cc40a2b80 msgr2=0x7f5cc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.896+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cc40a2b80 0x7f5cc40a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00ffdd0 tx=0x7f5cc8009260 comp rx=0 tx=0).stop 2024-09-12T03:25:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.896+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 msgr2=0x7f5ce019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.896+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 0x7f5ce019a570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0035040 tx=0x7f5cd0033d90 comp rx=0 tx=0).stop 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.896+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 shutdown_connections 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.896+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f5cc40a2b80 0x7f5cc40a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.897+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce0109090 0x7f5ce019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.897+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ce0104800 0x7f5ce019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.897+0000 7f5ce4e00640 1 --2- 172.21.15.50:0/838592314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ce0103e50 0x7f5ce019a570 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.897+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 >> 172.21.15.50:0/838592314 conn(0x7f5ce00ff8b0 msgr2=0x7f5ce0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.897+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 shutdown_connections 2024-09-12T03:25:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:03.897+0000 7f5ce4e00640 1 -- 172.21.15.50:0/838592314 wait complete. 2024-09-12T03:25:04.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:03 smithi142 ceph-mon[35329]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:04.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:03 smithi082 ceph-mon[35046]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:04.671 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:25:04.671 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:25:04.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:25:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:04 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/838592314' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:25:05.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:04 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/838592314' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:25:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:04 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/838592314' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:25:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 -- 172.21.15.50:0/743293119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb01079c0 msgr2=0x7fceb0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 --2- 172.21.15.50:0/743293119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb01079c0 0x7fceb0107da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009860 tx=0x7fcea002a340 comp rx=0 tx=0).stop 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 -- 172.21.15.50:0/743293119 shutdown_connections 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 --2- 172.21.15.50:0/743293119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb0102360 0x7fceb010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 --2- 172.21.15.50:0/743293119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb01019c0 0x7fceb0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 --2- 172.21.15.50:0/743293119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb01079c0 0x7fceb0107da0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 -- 172.21.15.50:0/743293119 >> 172.21.15.50:0/743293119 conn(0x7fceb00fd740 msgr2=0x7fceb00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 -- 172.21.15.50:0/743293119 shutdown_connections 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.710+0000 7fceafe00640 1 -- 172.21.15.50:0/743293119 wait complete. 2024-09-12T03:25:05.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.711+0000 7fceafe00640 1 Processor -- start 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 -- start start 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb01019c0 0x7fceb019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 0x7fceb019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb01079c0 0x7fceb019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0111dc0 con 0x7fceb01079c0 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0111c40 con 0x7fceb0102360 2024-09-12T03:25:05.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceafe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0111f40 con 0x7fceb01019c0 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceae000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 0x7fceb019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceaea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb01019c0 0x7fceb019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceae000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 0x7fceb019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:60648/0 (socket says 172.21.15.50:60648) 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceae000640 1 -- 172.21.15.50:0/1861649870 learned_addr learned my addr 172.21.15.50:0/1861649870 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.712+0000 7fceaf400640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb01079c0 0x7fceb019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceae000640 1 -- 172.21.15.50:0/1861649870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb01019c0 msgr2=0x7fceb019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceae000640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb01019c0 0x7fceb019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceae000640 1 -- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb01079c0 msgr2=0x7fceb019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceae000640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb01079c0 0x7fceb019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceae000640 1 -- 172.21.15.50:0/1861649870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb019f440 con 0x7fceb0102360 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceaf400640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb01079c0 0x7fceb019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:05.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceae000640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 0x7fceb019a9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e9f0 tx=0x7fce9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:05.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00cd80 con 0x7fceb0102360 2024-09-12T03:25:05.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c010910 con 0x7fceb0102360 2024-09-12T03:25:05.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c018990 con 0x7fceb0102360 2024-09-12T03:25:05.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.713+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb019f730 con 0x7fceb0102360 2024-09-12T03:25:05.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.714+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb01a7010 con 0x7fceb0102360 2024-09-12T03:25:05.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.715+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb0101e20 con 0x7fceb0102360 2024-09-12T03:25:05.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.717+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce9c010ac0 con 0x7fceb0102360 2024-09-12T03:25:05.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.718+0000 7fcea7400640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce940a2c00 0x7fce940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:05.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.718+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fce9c0cfa40 con 0x7fceb0102360 2024-09-12T03:25:05.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.718+0000 7fceaea00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce940a2c00 0x7fce940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:05.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.719+0000 7fceaea00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce940a2c00 0x7fce940a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcea002a930 tx=0x7fcea00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:05.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.720+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce9c09c700 con 0x7fceb0102360 2024-09-12T03:25:05.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:05 smithi050 ceph-mon[30378]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.938+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fceb019b7f0 con 0x7fceb0102360 2024-09-12T03:25:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.939+0000 7fcea7400640 1 -- 172.21.15.50:0/1861649870 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+592 (secure 0 0 0) 0x7fceb019b7f0 con 0x7fceb0102360 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3330091767 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2850856182 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1363145359 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3574998620 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2127938282 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/1939431272 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2850856182 2024-09-13T03:17:12.653017+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/1939431272 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1226501005 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:05.940 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1765792264 2024-09-13T03:16:45.524611+0000 2024-09-12T03:25:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-09-12T03:25:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce940a2c00 msgr2=0x7fce940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce940a2c00 0x7fce940a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcea002a930 tx=0x7fcea00058d0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 msgr2=0x7fceb019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 0x7fceb019a9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e9f0 tx=0x7fce9c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 shutdown_connections 2024-09-12T03:25:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce940a2c00 0x7fce940a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb01079c0 0x7fceb019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb0102360 0x7fceb019a9d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 --2- 172.21.15.50:0/1861649870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb01019c0 0x7fceb019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.941+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 >> 172.21.15.50:0/1861649870 conn(0x7fceb00fd740 msgr2=0x7fceb00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.942+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 shutdown_connections 2024-09-12T03:25:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:05.942+0000 7fceafe00640 1 -- 172.21.15.50:0/1861649870 wait complete. 2024-09-12T03:25:06.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:05 smithi142 ceph-mon[35329]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:05 smithi082 ceph-mon[35046]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:06.681 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2024-09-12T03:25:06.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:06.682 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:25:06.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[74734]: fuse finished with error 0 and tester_r 0 2024-09-12T03:25:06.742 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:25:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:06 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1861649870' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:25:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:06 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1861649870' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:25:07.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:06 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1861649870' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:25:08.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:07 smithi142 ceph-mon[35329]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-12T03:25:08.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:07 smithi082 ceph-mon[35046]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-12T03:25:08.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:07 smithi050 ceph-mon[30378]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-12T03:25:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:09 smithi082 ceph-mon[35046]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-12T03:25:10.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:09 smithi050 ceph-mon[30378]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-12T03:25:10.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:09 smithi142 ceph-mon[35329]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-12T03:25:12.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:11 smithi082 ceph-mon[35046]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:25:12.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:11 smithi050 ceph-mon[30378]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:25:12.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:11 smithi142 ceph-mon[35329]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:25:12.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:25:12.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:25:12.844 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-12T03:25:12.931 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:25:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-12T03:25:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T03:25:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-12T03:25:13.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:12 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]: dispatch 2024-09-12T03:25:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-12T03:25:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T03:25:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-09-12T03:25:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-12T03:25:13.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-12T03:25:13.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-09-12T03:25:13.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:25:13.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-12T03:25:13.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-12T03:25:13.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:12 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]: dispatch 2024-09-12T03:25:13.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-12T03:25:13.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:12 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]: dispatch 2024-09-12T03:25:13.313 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 -- 172.21.15.50:0/2068787064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550106f10 msgr2=0x7f955010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 --2- 172.21.15.50:0/2068787064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550106f10 0x7f955010ea90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9548009ca0 tx=0x7f954802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 -- 172.21.15.50:0/2068787064 shutdown_connections 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 --2- 172.21.15.50:0/2068787064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550106f10 0x7f955010ea90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 --2- 172.21.15.50:0/2068787064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9550102790 0x7f95501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 --2- 172.21.15.50:0/2068787064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550101de0 0x7f95501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 -- 172.21.15.50:0/2068787064 >> 172.21.15.50:0/2068787064 conn(0x7f95500fd8c0 msgr2=0x7f95500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 -- 172.21.15.50:0/2068787064 shutdown_connections 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.943+0000 7f9557800640 1 -- 172.21.15.50:0/2068787064 wait complete. 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.944+0000 7f9557800640 1 Processor -- start 2024-09-12T03:25:13.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.944+0000 7f9557800640 1 -- start start 2024-09-12T03:25:13.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9557800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 0x7f95500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9557800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550102790 0x7f9550074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9557800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9550106f10 0x7f9550074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9557800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9550111f70 con 0x7f9550101de0 2024-09-12T03:25:13.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9557800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9550111df0 con 0x7f9550102790 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9557800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95501120f0 con 0x7f9550106f10 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9554e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 0x7f95500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f954fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550102790 0x7f9550074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9554e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 0x7f95500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60974/0 (socket says 172.21.15.50:60974) 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9554e00640 1 -- 172.21.15.50:0/1053520267 learned_addr learned my addr 172.21.15.50:0/1053520267 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9555800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9550106f10 0x7f9550074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9554e00640 1 -- 172.21.15.50:0/1053520267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9550106f10 msgr2=0x7f9550074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9554e00640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9550106f10 0x7f9550074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.945+0000 7f9554e00640 1 -- 172.21.15.50:0/1053520267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550102790 msgr2=0x7f9550074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f9554e00640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550102790 0x7f9550074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f9554e00640 1 -- 172.21.15.50:0/1053520267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95501a6810 con 0x7f9550101de0 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f954fe00640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550102790 0x7f9550074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:13.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f9554e00640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 0x7f95500744f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f954400ea60 tx=0x7f954400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954400cd20 con 0x7f9550101de0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95501a6b00 con 0x7f9550101de0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95440035f0 con 0x7f9550101de0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544010690 con 0x7f9550101de0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.946+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95501a7040 con 0x7f9550101de0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.947+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95501021c0 con 0x7f9550101de0 2024-09-12T03:25:13.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.949+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f95440037a0 con 0x7f9550101de0 2024-09-12T03:25:13.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.950+0000 7f954d600640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f95380a2b80 0x7f95380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.951+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f95440cf720 con 0x7f9550101de0 2024-09-12T03:25:13.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.951+0000 7f954fe00640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f95380a2b80 0x7f95380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.952+0000 7f954fe00640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f95380a2b80 0x7f95380a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9550075f80 tx=0x7f953c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:13.955+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f954409c2c0 con 0x7f9550101de0 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:25:13.956 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-12T03:25:13.957 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-12T03:25:13.957 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]': finished 2024-09-12T03:25:13.957 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:13 smithi050 ceph-mon[30378]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:25:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:25:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-12T03:25:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T03:25:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-12T03:25:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-12T03:25:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-12T03:25:14.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-12T03:25:14.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:25:14.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-12T03:25:14.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-12T03:25:14.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]': finished 2024-09-12T03:25:14.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:13 smithi082 ceph-mon[35046]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:25:14.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.143+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9550075a20 con 0x7f9550101de0 2024-09-12T03:25:14.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.145+0000 7f954d600640 1 -- 172.21.15.50:0/1053520267 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24622 (secure 0 0 0) 0x7f954409bbc0 con 0x7f9550101de0 2024-09-12T03:25:14.146 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:14.147 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":92,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:25:13.804593+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f95380a2b80 msgr2=0x7f95380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f95380a2b80 0x7f95380a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9550075f80 tx=0x7f953c00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 msgr2=0x7f95500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 0x7f95500744f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f954400ea60 tx=0x7f954400ee60 comp rx=0 tx=0).stop 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 shutdown_connections 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f95380a2b80 0x7f95380a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9550106f10 0x7f9550074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9550102790 0x7f9550074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 --2- 172.21.15.50:0/1053520267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9550101de0 0x7f95500744f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 >> 172.21.15.50:0/1053520267 conn(0x7f95500fd8c0 msgr2=0x7f95500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 shutdown_connections 2024-09-12T03:25:14.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:14.148+0000 7f9557800640 1 -- 172.21.15.50:0/1053520267 wait complete. 2024-09-12T03:25:14.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 10]}]': finished 2024-09-12T03:25:14.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:13 smithi142 ceph-mon[35329]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:25:14.971 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:25:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:14 smithi082 ceph-mon[35046]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:25:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:14 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1053520267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:14 smithi050 ceph-mon[30378]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:25:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:14 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1053520267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:15.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:14 smithi142 ceph-mon[35329]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:25:15.588 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:14 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1053520267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:15.589 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:16.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 -- 172.21.15.50:0/2470360906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94077040 msgr2=0x7faa94112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 --2- 172.21.15.50:0/2470360906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94077040 0x7faa94112510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa80009900 tx=0x7faa8002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 -- 172.21.15.50:0/2470360906 shutdown_connections 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 --2- 172.21.15.50:0/2470360906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94112a50 0x7faa94114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 --2- 172.21.15.50:0/2470360906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94077040 0x7faa94112510 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 --2- 172.21.15.50:0/2470360906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94076720 0x7faa94076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.023+0000 7faa9c000640 1 -- 172.21.15.50:0/2470360906 >> 172.21.15.50:0/2470360906 conn(0x7faa940ffa50 msgr2=0x7faa94101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.024+0000 7faa9c000640 1 -- 172.21.15.50:0/2470360906 shutdown_connections 2024-09-12T03:25:16.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.024+0000 7faa9c000640 1 -- 172.21.15.50:0/2470360906 wait complete. 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.025+0000 7faa9c000640 1 Processor -- start 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.025+0000 7faa9c000640 1 -- start start 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.025+0000 7faa9c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94076720 0x7faa941a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.025+0000 7faa9c000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94077040 0x7faa941a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 0x7faa9419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa941185e0 con 0x7faa94112a50 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9c000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa94118460 con 0x7faa94077040 2024-09-12T03:25:16.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa94118760 con 0x7faa94076720 2024-09-12T03:25:16.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 0x7faa9419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:16.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa98c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94077040 0x7faa941a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:16.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa99600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94076720 0x7faa941a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:16.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 0x7faa9419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60986/0 (socket says 172.21.15.50:60986) 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 -- 172.21.15.50:0/2811345782 learned_addr learned my addr 172.21.15.50:0/2811345782 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 -- 172.21.15.50:0/2811345782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94076720 msgr2=0x7faa941a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94076720 0x7faa941a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 -- 172.21.15.50:0/2811345782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94077040 msgr2=0x7faa941a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94077040 0x7faa941a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.026+0000 7faa9a000640 1 -- 172.21.15.50:0/2811345782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9419f060 con 0x7faa94112a50 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa98c00640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94077040 0x7faa941a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:16.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa9a000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 0x7faa9419e930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00dac0 tx=0x7faa8c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:16.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c017030 con 0x7faa94112a50 2024-09-12T03:25:16.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa8c003e50 con 0x7faa94112a50 2024-09-12T03:25:16.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9419f350 con 0x7faa94112a50 2024-09-12T03:25:16.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c010460 con 0x7faa94112a50 2024-09-12T03:25:16.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.027+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941ab640 con 0x7faa94112a50 2024-09-12T03:25:16.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.028+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa94076b00 con 0x7faa94112a50 2024-09-12T03:25:16.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.030+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa8c01a070 con 0x7faa94112a50 2024-09-12T03:25:16.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.031+0000 7faa92000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faa7c0a2c00 0x7faa7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:16.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.032+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faa8c0cf060 con 0x7faa94112a50 2024-09-12T03:25:16.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.032+0000 7faa99600640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faa7c0a2c00 0x7faa7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:16.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.033+0000 7faa99600640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faa7c0a2c00 0x7faa7c0a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa9419fa70 tx=0x7faa88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:16.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.036+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa8c09bc00 con 0x7faa94112a50 2024-09-12T03:25:16.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:15 smithi082 ceph-mon[35046]: pgmap v323: 129 pgs: 1 activating, 4 peering, 124 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-12T03:25:16.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:15 smithi050 ceph-mon[30378]: pgmap v323: 129 pgs: 1 activating, 4 peering, 124 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-12T03:25:16.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.225+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa9402d7c0 con 0x7faa94112a50 2024-09-12T03:25:16.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.227+0000 7faa92000640 1 -- 172.21.15.50:0/2811345782 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24622 (secure 0 0 0) 0x7faa8c09b500 con 0x7faa94112a50 2024-09-12T03:25:16.228 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:16.228 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":92,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:25:13.804593+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:25:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faa7c0a2c00 msgr2=0x7faa7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faa7c0a2c00 0x7faa7c0a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa9419fa70 tx=0x7faa88005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 msgr2=0x7faa9419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 0x7faa9419e930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00dac0 tx=0x7faa8c00df80 comp rx=0 tx=0).stop 2024-09-12T03:25:16.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 shutdown_connections 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7faa7c0a2c00 0x7faa7c0a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa94112a50 0x7faa9419e930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa94077040 0x7faa941a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 --2- 172.21.15.50:0/2811345782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94076720 0x7faa941a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 >> 172.21.15.50:0/2811345782 conn(0x7faa940ffa50 msgr2=0x7faa94101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.229+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 shutdown_connections 2024-09-12T03:25:16.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:16.230+0000 7faa9c000640 1 -- 172.21.15.50:0/2811345782 wait complete. 2024-09-12T03:25:16.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:15 smithi142 ceph-mon[35329]: pgmap v323: 129 pgs: 1 activating, 4 peering, 124 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-12T03:25:16.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:16 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2811345782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:17.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:25:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:16 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2811345782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:17.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:16 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2811345782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:17.458 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:17.869 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:17 smithi050 ceph-mon[30378]: pgmap v324: 129 pgs: 1 activating, 4 peering, 124 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-12T03:25:17.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 -- 172.21.15.50:0/3653597803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 msgr2=0x7fbe54110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:17.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 --2- 172.21.15.50:0/3653597803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe54110ae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009ca0 tx=0x7fbe4c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:25:17.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 -- 172.21.15.50:0/3653597803 shutdown_connections 2024-09-12T03:25:17.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 --2- 172.21.15.50:0/3653597803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe54110ae0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:17.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 --2- 172.21.15.50:0/3653597803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 0x7fbe54108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:17.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 --2- 172.21.15.50:0/3653597803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe54104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:17.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 -- 172.21.15.50:0/3653597803 >> 172.21.15.50:0/3653597803 conn(0x7fbe540ff910 msgr2=0x7fbe54101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:17.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 -- 172.21.15.50:0/3653597803 shutdown_connections 2024-09-12T03:25:17.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.985+0000 7fbe5a600640 1 -- 172.21.15.50:0/3653597803 wait complete. 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.986+0000 7fbe5a600640 1 Processor -- start 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 -- start start 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe5419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 0x7fbe5419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe5419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54114010 con 0x7fbe54108f60 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54113e90 con 0x7fbe541047e0 2024-09-12T03:25:17.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.987+0000 7fbe5a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54114190 con 0x7fbe54103e30 2024-09-12T03:25:17.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe59200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe5419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe59200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe5419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:33236/0 (socket says 172.21.15.50:33236) 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe59200640 1 -- 172.21.15.50:0/2232559805 learned_addr learned my addr 172.21.15.50:0/2232559805 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe59c00640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe5419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 0x7fbe5419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 -- 172.21.15.50:0/2232559805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 msgr2=0x7fbe5419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe5419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 -- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 msgr2=0x7fbe5419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe5419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 -- 172.21.15.50:0/2232559805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5419f460 con 0x7fbe541047e0 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe59c00640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe5419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe59200640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe5419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:17.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.988+0000 7fbe53e00640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 0x7fbe5419a9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e9f0 tx=0x7fbe4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:17.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.989+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000ccc0 con 0x7fbe541047e0 2024-09-12T03:25:17.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.989+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5419f750 con 0x7fbe541047e0 2024-09-12T03:25:17.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.989+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe40010910 con 0x7fbe541047e0 2024-09-12T03:25:17.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.989+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40018960 con 0x7fbe541047e0 2024-09-12T03:25:17.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.989+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe541a7030 con 0x7fbe541047e0 2024-09-12T03:25:17.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.990+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe54105310 con 0x7fbe541047e0 2024-09-12T03:25:17.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.993+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbe40010ac0 con 0x7fbe541047e0 2024-09-12T03:25:17.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.994+0000 7fbe51600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:17.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.994+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbe400cfad0 con 0x7fbe541047e0 2024-09-12T03:25:17.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.994+0000 7fbe59200640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:17.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.995+0000 7fbe59200640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe48005fd0 tx=0x7fbe4800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:17.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:17.995+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe4009c020 con 0x7fbe541047e0 2024-09-12T03:25:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:17 smithi082 ceph-mon[35046]: pgmap v324: 129 pgs: 1 activating, 4 peering, 124 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-12T03:25:18.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.222+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbe54104210 con 0x7fbe541047e0 2024-09-12T03:25:18.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:17 smithi142 ceph-mon[35329]: pgmap v324: 129 pgs: 1 activating, 4 peering, 124 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-12T03:25:18.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.836+0000 7fbe51600640 1 -- 172.21.15.50:0/2232559805 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fbe54104210 con 0x7fbe541047e0 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbe3c0a2b80 msgr2=0x7fbe3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe48005fd0 tx=0x7fbe4800b480 comp rx=0 tx=0).stop 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 msgr2=0x7fbe5419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 0x7fbe5419a9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e9f0 tx=0x7fbe4000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 shutdown_connections 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe54108f60 0x7fbe5419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe541047e0 0x7fbe5419a9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 --2- 172.21.15.50:0/2232559805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe54103e30 0x7fbe5419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:18.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 >> 172.21.15.50:0/2232559805 conn(0x7fbe540ff910 msgr2=0x7fbe54100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:18.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 shutdown_connections 2024-09-12T03:25:18.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:18.839+0000 7fbe5a600640 1 -- 172.21.15.50:0/2232559805 wait complete. 2024-09-12T03:25:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:18 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2232559805' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:25:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:18 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:25:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:18 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2232559805' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:25:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:18 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:25:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:18 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2232559805' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:25:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:18 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:25:19.552 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-12T03:25:19.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T03:25:19.939 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:20.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:19 smithi082 ceph-mon[35046]: pgmap v325: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:20.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:19 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:25:20.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:19 smithi082 ceph-mon[35046]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:25:20.107 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:19 smithi050 ceph-mon[30378]: pgmap v325: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:20.107 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:19 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:25:20.107 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:19 smithi050 ceph-mon[30378]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:25:20.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:19 smithi142 ceph-mon[35329]: pgmap v325: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:20.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:19 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:25:20.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:19 smithi142 ceph-mon[35329]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:25:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 -- 172.21.15.50:0/737253412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 msgr2=0x7f9168101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 --2- 172.21.15.50:0/737253412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f9168101e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9154009930 tx=0x7f915402a420 comp rx=0 tx=0).stop 2024-09-12T03:25:20.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 -- 172.21.15.50:0/737253412 shutdown_connections 2024-09-12T03:25:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 --2- 172.21.15.50:0/737253412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9168102360 0x7f916810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 --2- 172.21.15.50:0/737253412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f9168101e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 --2- 172.21.15.50:0/737253412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681079c0 0x7f9168107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 -- 172.21.15.50:0/737253412 >> 172.21.15.50:0/737253412 conn(0x7f91680fd740 msgr2=0x7f91680ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 -- 172.21.15.50:0/737253412 shutdown_connections 2024-09-12T03:25:20.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.475+0000 7f916de00640 1 -- 172.21.15.50:0/737253412 wait complete. 2024-09-12T03:25:20.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.476+0000 7f916de00640 1 Processor -- start 2024-09-12T03:25:20.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.476+0000 7f916de00640 1 -- start start 2024-09-12T03:25:20.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f916819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:20.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9168102360 0x7f916819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:20.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681079c0 0x7f916819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111dc0 con 0x7f91681019c0 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111c40 con 0x7f91681079c0 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111f40 con 0x7f9168102360 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f916819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f916819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46988/0 (socket says 172.21.15.50:46988) 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916ca00640 1 -- 172.21.15.50:0/816077394 learned_addr learned my addr 172.21.15.50:0/816077394 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f9167e00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9168102360 0x7f916819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.477+0000 7f916d400640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681079c0 0x7f916819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916ca00640 1 -- 172.21.15.50:0/816077394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9168102360 msgr2=0x7f916819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916ca00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9168102360 0x7f916819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916ca00640 1 -- 172.21.15.50:0/816077394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681079c0 msgr2=0x7f916819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916ca00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681079c0 0x7f916819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916ca00640 1 -- 172.21.15.50:0/816077394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916819f460 con 0x7f91681019c0 2024-09-12T03:25:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916ca00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f916819a4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e9f0 tx=0x7f915c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c00cd80 con 0x7f91681019c0 2024-09-12T03:25:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916819f750 con 0x7f91681019c0 2024-09-12T03:25:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c010910 con 0x7f91681019c0 2024-09-12T03:25:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c0188f0 con 0x7f91681019c0 2024-09-12T03:25:20.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.478+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91681a7030 con 0x7f91681019c0 2024-09-12T03:25:20.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.480+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9168101e20 con 0x7f91681019c0 2024-09-12T03:25:20.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.481+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f915c010ac0 con 0x7f91681019c0 2024-09-12T03:25:20.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.484+0000 7f9165600640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f91500a2c00 0x7f91500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.484+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f915c0cf680 con 0x7f91681019c0 2024-09-12T03:25:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.484+0000 7f9167e00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f91500a2c00 0x7f91500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:20.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.485+0000 7f9167e00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f91500a2c00 0x7f91500a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f91540058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:20.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.488+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f915c09bbd0 con 0x7f91681019c0 2024-09-12T03:25:20.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.712+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f91681a72c0 con 0x7f91681019c0 2024-09-12T03:25:20.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.841+0000 7f9165600640 1 -- 172.21.15.50:0/816077394 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f915c09fed0 con 0x7f91681019c0 2024-09-12T03:25:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f91500a2c00 msgr2=0x7f91500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f91500a2c00 0x7f91500a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f91540058d0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 msgr2=0x7f916819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f916819a4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e9f0 tx=0x7f915c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 shutdown_connections 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f91500a2c00 0x7f91500a50c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681079c0 0x7f916819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9168102360 0x7f916819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 --2- 172.21.15.50:0/816077394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91681019c0 0x7f916819a4b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:20.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 >> 172.21.15.50:0/816077394 conn(0x7f91680fd740 msgr2=0x7f91680fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:20.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 shutdown_connections 2024-09-12T03:25:20.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:20.844+0000 7f916de00640 1 -- 172.21.15.50:0/816077394 wait complete. 2024-09-12T03:25:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:20 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/816077394' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:25:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:20 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/816077394' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:25:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:20 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/816077394' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:25:21.561 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T03:25:21.945 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:25:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/816077394' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:25:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:25:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:25:22.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:25:22.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:21 smithi082 ceph-mon[35046]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:25:22.107 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:22.107 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:25:22.108 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/816077394' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:25:22.108 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:25:22.108 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:25:22.108 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:25:22.108 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:21 smithi050 ceph-mon[30378]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:25:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:25:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/816077394' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:25:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:25:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:25:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:25:22.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:21 smithi142 ceph-mon[35329]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 -- 172.21.15.50:0/119689304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3268104010 msgr2=0x7f32681043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 --2- 172.21.15.50:0/119689304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3268104010 0x7f32681043f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3258009900 tx=0x7f325802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 -- 172.21.15.50:0/119689304 shutdown_connections 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 --2- 172.21.15.50:0/119689304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3268109140 0x7f3268110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 --2- 172.21.15.50:0/119689304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32681049c0 0x7f3268108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 --2- 172.21.15.50:0/119689304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3268104010 0x7f32681043f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 -- 172.21.15.50:0/119689304 >> 172.21.15.50:0/119689304 conn(0x7f32680ffa50 msgr2=0x7f3268101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 -- 172.21.15.50:0/119689304 shutdown_connections 2024-09-12T03:25:22.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.517+0000 7f326f800640 1 -- 172.21.15.50:0/119689304 wait complete. 2024-09-12T03:25:22.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.518+0000 7f326f800640 1 Processor -- start 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.518+0000 7f326f800640 1 -- start start 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.518+0000 7f326f800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 0x7f326819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.518+0000 7f326f800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32681049c0 0x7f326819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3268109140 0x7f326819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326f800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3268114190 con 0x7f32681049c0 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3268114010 con 0x7f3268109140 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326f800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3268114310 con 0x7f3268104010 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f325fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32681049c0 0x7f326819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 0x7f326819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3268109140 0x7f326819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 0x7f326819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41902/0 (socket says 172.21.15.50:41902) 2024-09-12T03:25:22.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 -- 172.21.15.50:0/3293691823 learned_addr learned my addr 172.21.15.50:0/3293691823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:22.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 -- 172.21.15.50:0/3293691823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3268109140 msgr2=0x7f326819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:22.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3268109140 0x7f326819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 -- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32681049c0 msgr2=0x7f326819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:22.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32681049c0 0x7f326819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f326ce00640 1 -- 172.21.15.50:0/3293691823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326819f5b0 con 0x7f3268104010 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.519+0000 7f325fe00640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32681049c0 0x7f326819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.520+0000 7f326ce00640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 0x7f326819a600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3258004770 tx=0x7f3258003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.520+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325803f070 con 0x7f3268104010 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.520+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326819f840 con 0x7f3268104010 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.520+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3258031ca0 con 0x7f3268104010 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.520+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258035030 con 0x7f3268104010 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.520+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681a7180 con 0x7f3268104010 2024-09-12T03:25:22.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.521+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32681043f0 con 0x7f3268104010 2024-09-12T03:25:22.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.523+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3258042070 con 0x7f3268104010 2024-09-12T03:25:22.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.524+0000 7f3266000640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f324c0a2b80 0x7f324c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:22.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.525+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32580effe0 con 0x7f3268104010 2024-09-12T03:25:22.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.525+0000 7f325fe00640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f324c0a2b80 0x7f324c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:22.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.525+0000 7f325fe00640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f324c0a2b80 0x7f324c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32680ffe70 tx=0x7f325000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:22.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.527+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32580bc590 con 0x7f3268104010 2024-09-12T03:25:22.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.749+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f326810c320 con 0x7f3268104010 2024-09-12T03:25:22.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.859+0000 7f3266000640 1 -- 172.21.15.50:0/3293691823 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f32580c0890 con 0x7f3268104010 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f324c0a2b80 msgr2=0x7f324c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f324c0a2b80 0x7f324c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32680ffe70 tx=0x7f325000b410 comp rx=0 tx=0).stop 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 msgr2=0x7f326819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 0x7f326819a600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3258004770 tx=0x7f3258003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 shutdown_connections 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f324c0a2b80 0x7f324c0a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3268109140 0x7f326819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32681049c0 0x7f326819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 --2- 172.21.15.50:0/3293691823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268104010 0x7f326819a600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 >> 172.21.15.50:0/3293691823 conn(0x7f32680ffa50 msgr2=0x7f3268100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 shutdown_connections 2024-09-12T03:25:22.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:22.862+0000 7f326f800640 1 -- 172.21.15.50:0/3293691823 wait complete. 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] up:active 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: Cluster is now healthy 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T03:25:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T03:25:23.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3293691823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:25:23.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:22 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:25:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] up:active 2024-09-12T03:25:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:25:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:25:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: Cluster is now healthy 2024-09-12T03:25:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:25:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T03:25:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T03:25:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3293691823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:25:23.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:22 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:25:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] up:active 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: Cluster is now healthy 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3293691823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:25:23.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:22 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:25:23.568 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T03:25:23.949 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:23 smithi050 ceph-mon[30378]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:23 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:25:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:23 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] up:active 2024-09-12T03:25:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:23 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-12T03:25:24.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:23 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:25:24.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:23 smithi142 ceph-mon[35329]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:24.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:23 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:25:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:23 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] up:active 2024-09-12T03:25:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:23 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-12T03:25:24.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:23 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:25:24.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:23 smithi082 ceph-mon[35046]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:25:24.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:23 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:25:24.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:23 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] up:active 2024-09-12T03:25:24.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:23 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-12T03:25:24.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:23 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:25:24.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 -- 172.21.15.50:0/3007464274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 msgr2=0x7fc494108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:24.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 --2- 172.21.15.50:0/3007464274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc494108a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009900 tx=0x7fc47c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 -- 172.21.15.50:0/3007464274 shutdown_connections 2024-09-12T03:25:24.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 --2- 172.21.15.50:0/3007464274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc494109140 0x7fc494110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 --2- 172.21.15.50:0/3007464274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc494108a10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 --2- 172.21.15.50:0/3007464274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc494104010 0x7fc4941043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 -- 172.21.15.50:0/3007464274 >> 172.21.15.50:0/3007464274 conn(0x7fc4940ffa50 msgr2=0x7fc494101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:24.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 -- 172.21.15.50:0/3007464274 shutdown_connections 2024-09-12T03:25:24.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.489+0000 7fc49ae00640 1 -- 172.21.15.50:0/3007464274 wait complete. 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 Processor -- start 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 -- start start 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc494104010 0x7fc49419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc49419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc494109140 0x7fc49419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc494114160 con 0x7fc4941049c0 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc494113fe0 con 0x7fc494109140 2024-09-12T03:25:24.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.490+0000 7fc49ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4941142e0 con 0x7fc494104010 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc49419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc49419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47016/0 (socket says 172.21.15.50:47016) 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 -- 172.21.15.50:0/455674045 learned_addr learned my addr 172.21.15.50:0/455674045 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc498e00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc494109140 0x7fc49419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc493e00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc494104010 0x7fc49419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 -- 172.21.15.50:0/455674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc494104010 msgr2=0x7fc49419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc494104010 0x7fc49419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 -- 172.21.15.50:0/455674045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc494109140 msgr2=0x7fc49419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc494109140 0x7fc49419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.491+0000 7fc48be00640 1 -- 172.21.15.50:0/455674045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49419f690 con 0x7fc4941049c0 2024-09-12T03:25:24.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.492+0000 7fc48be00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc49419ac20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006010 tx=0x7fc47c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.492+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c03f030 con 0x7fc4941049c0 2024-09-12T03:25:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.492+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc47c004440 con 0x7fc4941049c0 2024-09-12T03:25:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.492+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c035070 con 0x7fc4941049c0 2024-09-12T03:25:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.492+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49419f920 con 0x7fc4941049c0 2024-09-12T03:25:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.492+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49419fdb0 con 0x7fc4941049c0 2024-09-12T03:25:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.493+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4941043f0 con 0x7fc4941049c0 2024-09-12T03:25:24.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.495+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc47c042030 con 0x7fc4941049c0 2024-09-12T03:25:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.496+0000 7fc491600640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc4780a2c00 0x7fc4780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.496+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc47c0ef6c0 con 0x7fc4941049c0 2024-09-12T03:25:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.497+0000 7fc493e00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc4780a2c00 0x7fc4780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:24.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.497+0000 7fc493e00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc4780a2c00 0x7fc4780a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc484009bd0 tx=0x7fc484005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:24.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.501+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc47c0bbc10 con 0x7fc4941049c0 2024-09-12T03:25:24.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.723+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc49410bdf0 con 0x7fc4941049c0 2024-09-12T03:25:24.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.863+0000 7fc491600640 1 -- 172.21.15.50:0/455674045 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fc47c0bff10 con 0x7fc4941049c0 2024-09-12T03:25:24.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc4780a2c00 msgr2=0x7fc4780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:24.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc4780a2c00 0x7fc4780a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc484009bd0 tx=0x7fc484005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:24.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 msgr2=0x7fc49419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:24.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc49419ac20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006010 tx=0x7fc47c004040 comp rx=0 tx=0).stop 2024-09-12T03:25:24.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 shutdown_connections 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc4780a2c00 0x7fc4780a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc494109140 0x7fc49419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4941049c0 0x7fc49419ac20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 --2- 172.21.15.50:0/455674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc494104010 0x7fc49419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 >> 172.21.15.50:0/455674045 conn(0x7fc4940ffa50 msgr2=0x7fc494100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.867+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 shutdown_connections 2024-09-12T03:25:24.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:24.868+0000 7fc49ae00640 1 -- 172.21.15.50:0/455674045 wait complete. 2024-09-12T03:25:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:24 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/455674045' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:25:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:24 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/455674045' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:25:25.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:24 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/455674045' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:25:25.549 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-12T03:25:25.549 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T03:25:25.928 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:25 smithi050 ceph-mon[30378]: pgmap v328: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:25:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:25 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/455674045' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:25:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:25 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:25:26.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:25 smithi142 ceph-mon[35329]: pgmap v328: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:25:26.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:25 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/455674045' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:25:26.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:25 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:25:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:25 smithi082 ceph-mon[35046]: pgmap v328: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:25:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:25 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/455674045' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:25:26.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:25 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:25:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.461+0000 7f44c5000640 1 -- 172.21.15.50:0/4271423314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 msgr2=0x7f44c01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.461+0000 7f44c5000640 1 --2- 172.21.15.50:0/4271423314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 0x7f44c01089f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009900 tx=0x7f44a802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 -- 172.21.15.50:0/4271423314 shutdown_connections 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 --2- 172.21.15.50:0/4271423314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0109120 0x7f44c0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 --2- 172.21.15.50:0/4271423314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 0x7f44c01089f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 --2- 172.21.15.50:0/4271423314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c0103ff0 0x7f44c01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 -- 172.21.15.50:0/4271423314 >> 172.21.15.50:0/4271423314 conn(0x7f44c00ffa90 msgr2=0x7f44c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 -- 172.21.15.50:0/4271423314 shutdown_connections 2024-09-12T03:25:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.462+0000 7f44c5000640 1 -- 172.21.15.50:0/4271423314 wait complete. 2024-09-12T03:25:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 Processor -- start 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 -- start start 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 0x7f44c019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 0x7f44c019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c0109120 0x7f44c019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0114190 con 0x7f44c01049a0 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0114010 con 0x7f44c0103ff0 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.463+0000 7f44c5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0114310 con 0x7f44c0109120 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 0x7f44c019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 0x7f44c019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39698/0 (socket says 172.21.15.50:39698) 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 -- 172.21.15.50:0/2447123367 learned_addr learned my addr 172.21.15.50:0/2447123367 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44b5400640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 0x7f44c019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44be800640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c0109120 0x7f44c019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 -- 172.21.15.50:0/2447123367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c0109120 msgr2=0x7f44c019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c0109120 0x7f44c019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 -- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 msgr2=0x7f44c019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 0x7f44c019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.464+0000 7f44bde00640 1 -- 172.21.15.50:0/2447123367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c019f5c0 con 0x7f44c0103ff0 2024-09-12T03:25:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.465+0000 7f44bde00640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 0x7f44c019a610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e9f0 tx=0x7f44b000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.465+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000cd80 con 0x7f44c0103ff0 2024-09-12T03:25:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.465+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c019f8b0 con 0x7f44c0103ff0 2024-09-12T03:25:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.465+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b0005500 con 0x7f44c0103ff0 2024-09-12T03:25:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.465+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c01a7190 con 0x7f44c0103ff0 2024-09-12T03:25:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.465+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0010690 con 0x7f44c0103ff0 2024-09-12T03:25:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.467+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449c005290 con 0x7f44c0103ff0 2024-09-12T03:25:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.468+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44b00046c0 con 0x7f44c0103ff0 2024-09-12T03:25:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.469+0000 7f44b7400640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a40a2b80 0x7f44a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.470+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f44b00cf690 con 0x7f44c0103ff0 2024-09-12T03:25:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.470+0000 7f44b5400640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a40a2b80 0x7f44a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.470+0000 7f44b5400640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a40a2b80 0x7f44a40a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44a8000c00 tx=0x7f44a8005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:26.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.474+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44b009c230 con 0x7f44c0103ff0 2024-09-12T03:25:26.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.684+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f449c000bc0 con 0x7f44a40a2b80 2024-09-12T03:25:26.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.712+0000 7f44b7400640 1 -- 172.21.15.50:0/2447123367 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f449c000bc0 con 0x7f44a40a2b80 2024-09-12T03:25:26.712 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:25:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:25:26.691+0000 7f449c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:25:26.712 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:25:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:25:26.691+0000 7f449c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:25:26.712 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:25:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:25:26.691+0000 7f449c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:25:26.712 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:25:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:25:26.691+0000 7f449c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:25:26.713 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:25:26 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:25:26.691+0000 7f449c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:25:26.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.715+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a40a2b80 msgr2=0x7f44a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:26.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.715+0000 7f44c5000640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a40a2b80 0x7f44a40a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44a8000c00 tx=0x7f44a8005b50 comp rx=0 tx=0).stop 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.715+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 msgr2=0x7f44c019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.715+0000 7f44c5000640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 0x7f44c019a610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e9f0 tx=0x7f44b000edf0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 shutdown_connections 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44a40a2b80 0x7f44a40a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44c0109120 0x7f44c019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44c01049a0 0x7f44c019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 --2- 172.21.15.50:0/2447123367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44c0103ff0 0x7f44c019a610 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 >> 172.21.15.50:0/2447123367 conn(0x7f44c00ffa90 msgr2=0x7f44c0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 shutdown_connections 2024-09-12T03:25:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:26.716+0000 7f44c5000640 1 -- 172.21.15.50:0/2447123367 wait complete. 2024-09-12T03:25:27.084 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:26 smithi050 ceph-mon[30378]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:27.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:26 smithi142 ceph-mon[35329]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:27.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:26 smithi082 ceph-mon[35046]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:27.422 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T03:25:27.874 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:27 smithi142 ceph-mon[35329]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T03:25:27.874 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:27 smithi142 ceph-mon[35329]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T03:25:27.874 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:27 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:27 smithi082 ceph-mon[35046]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T03:25:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:27 smithi082 ceph-mon[35046]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T03:25:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:27 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:27 smithi050 ceph-mon[30378]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T03:25:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:27 smithi050 ceph-mon[30378]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T03:25:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:27 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:28.471 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.059+0000 7efc34800640 1 -- 172.21.15.50:0/2734311794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c109120 msgr2=0x7efc2c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.059+0000 7efc34800640 1 --2- 172.21.15.50:0/2734311794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c109120 0x7efc2c110ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc28009ca0 tx=0x7efc2802a8b0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 -- 172.21.15.50:0/2734311794 shutdown_connections 2024-09-12T03:25:29.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 --2- 172.21.15.50:0/2734311794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c109120 0x7efc2c110ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 --2- 172.21.15.50:0/2734311794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c1049a0 0x7efc2c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 --2- 172.21.15.50:0/2734311794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 -- 172.21.15.50:0/2734311794 >> 172.21.15.50:0/2734311794 conn(0x7efc2c0ffa90 msgr2=0x7efc2c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 -- 172.21.15.50:0/2734311794 shutdown_connections 2024-09-12T03:25:29.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.060+0000 7efc34800640 1 -- 172.21.15.50:0/2734311794 wait complete. 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.061+0000 7efc34800640 1 Processor -- start 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.061+0000 7efc34800640 1 -- start start 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc34800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc34800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c1049a0 0x7efc2c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc34800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c109120 0x7efc2c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc34800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c114150 con 0x7efc2c1049a0 2024-09-12T03:25:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc34800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c113fd0 con 0x7efc2c109120 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc34800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1142d0 con 0x7efc2c103ff0 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc31400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c1049a0 0x7efc2c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc31e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc31e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58792/0 (socket says 172.21.15.50:58792) 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc31e00640 1 -- 172.21.15.50:0/3764473321 learned_addr learned my addr 172.21.15.50:0/3764473321 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc31400640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c1049a0 0x7efc2c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44754/0 (socket says 172.21.15.50:44754) 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.062+0000 7efc32800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c109120 0x7efc2c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc31e00640 1 -- 172.21.15.50:0/3764473321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c109120 msgr2=0x7efc2c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc31e00640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c109120 0x7efc2c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc31e00640 1 -- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c1049a0 msgr2=0x7efc2c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc31e00640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c1049a0 0x7efc2c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc31e00640 1 -- 172.21.15.50:0/3764473321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2c19f5e0 con 0x7efc2c103ff0 2024-09-12T03:25:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc32800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c109120 0x7efc2c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc31e00640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c19a630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e9f0 tx=0x7efc2000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:29.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2000cd80 con 0x7efc2c103ff0 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20005500 con 0x7efc2c103ff0 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2c19f8d0 con 0x7efc2c103ff0 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.063+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20010690 con 0x7efc2c103ff0 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.064+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2c1a71b0 con 0x7efc2c103ff0 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.065+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc2c1043d0 con 0x7efc2c103ff0 2024-09-12T03:25:29.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.068+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc200046c0 con 0x7efc2c103ff0 2024-09-12T03:25:29.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.070+0000 7efc26a00640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efc140a2c10 0x7efc140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.070+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efc200cf970 con 0x7efc2c103ff0 2024-09-12T03:25:29.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.070+0000 7efc31400640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efc140a2c10 0x7efc140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.071+0000 7efc31400640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efc140a2c10 0x7efc140a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc2c10a9e0 tx=0x7efc1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:29.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.073+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc2009c3b0 con 0x7efc2c103ff0 2024-09-12T03:25:29.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.294+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efc2c100cc0 con 0x7efc140a2c10 2024-09-12T03:25:29.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.321+0000 7efc26a00640 1 -- 172.21.15.50:0/3764473321 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc2c100cc0 con 0x7efc140a2c10 2024-09-12T03:25:29.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efc140a2c10 msgr2=0x7efc140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efc140a2c10 0x7efc140a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc2c10a9e0 tx=0x7efc1c005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 msgr2=0x7efc2c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c19a630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e9f0 tx=0x7efc2000edf0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 shutdown_connections 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7efc140a2c10 0x7efc140a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c109120 0x7efc2c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc2c1049a0 0x7efc2c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 --2- 172.21.15.50:0/3764473321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc2c103ff0 0x7efc2c19a630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 >> 172.21.15.50:0/3764473321 conn(0x7efc2c0ffa90 msgr2=0x7efc2c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 shutdown_connections 2024-09-12T03:25:29.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:29.324+0000 7efc34800640 1 -- 172.21.15.50:0/3764473321 wait complete. 2024-09-12T03:25:29.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:29 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:29 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:29 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:29 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:29 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:29 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:29 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:29 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:29.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:29 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:30.060 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T03:25:30.454 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:30.769 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:30 smithi050 ceph-mon[30378]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T03:25:30.769 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:30 smithi050 ceph-mon[30378]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:30.769 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:30.769 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:30.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:30 smithi142 ceph-mon[35329]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T03:25:30.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:30 smithi142 ceph-mon[35329]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:30.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:30.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:30 smithi082 ceph-mon[35046]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T03:25:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:30 smithi082 ceph-mon[35046]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:30.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:31.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.002+0000 7febfea00640 1 -- 172.21.15.50:0/424585436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 msgr2=0x7febf81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.002+0000 7febfea00640 1 --2- 172.21.15.50:0/424585436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81124f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febe4009900 tx=0x7febe402a410 comp rx=0 tx=0).stop 2024-09-12T03:25:31.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 -- 172.21.15.50:0/424585436 shutdown_connections 2024-09-12T03:25:31.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 --2- 172.21.15.50:0/424585436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf8112a30 0x7febf8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 --2- 172.21.15.50:0/424585436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81124f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 --2- 172.21.15.50:0/424585436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 0x7febf8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 -- 172.21.15.50:0/424585436 >> 172.21.15.50:0/424585436 conn(0x7febf80ffa90 msgr2=0x7febf8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 -- 172.21.15.50:0/424585436 shutdown_connections 2024-09-12T03:25:31.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 -- 172.21.15.50:0/424585436 wait complete. 2024-09-12T03:25:31.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.003+0000 7febfea00640 1 Processor -- start 2024-09-12T03:25:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 -- start start 2024-09-12T03:25:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 0x7febf81a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf8112a30 0x7febf819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7febf81185f0 con 0x7febf8077040 2024-09-12T03:25:31.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febf8118470 con 0x7febf8112a30 2024-09-12T03:25:31.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7febf8118770 con 0x7febf8076720 2024-09-12T03:25:31.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febf7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febf7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 0x7febf81a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febf7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44786/0 (socket says 172.21.15.50:44786) 2024-09-12T03:25:31.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febf7400640 1 -- 172.21.15.50:0/837673349 learned_addr learned my addr 172.21.15.50:0/837673349 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:31.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.004+0000 7febfca00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf8112a30 0x7febf819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7400640 1 -- 172.21.15.50:0/837673349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 msgr2=0x7febf81a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7400640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 0x7febf81a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7400640 1 -- 172.21.15.50:0/837673349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf8112a30 msgr2=0x7febf819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7400640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf8112a30 0x7febf819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7400640 1 -- 172.21.15.50:0/837673349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf819f010 con 0x7febf8077040 2024-09-12T03:25:31.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7e00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 0x7febf81a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:31.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.005+0000 7febf7400640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81a4c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7febe4004770 tx=0x7febe4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.006+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe403f070 con 0x7febf8077040 2024-09-12T03:25:31.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.006+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf819f2a0 con 0x7febf8077040 2024-09-12T03:25:31.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.006+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe4031ca0 con 0x7febf8077040 2024-09-12T03:25:31.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.006+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4035030 con 0x7febf8077040 2024-09-12T03:25:31.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.006+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf81ab5f0 con 0x7febf8077040 2024-09-12T03:25:31.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.007+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf8076b00 con 0x7febf8077040 2024-09-12T03:25:31.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.008+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7febe4042070 con 0x7febf8077040 2024-09-12T03:25:31.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.010+0000 7febf4c00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7febdc0a2c10 0x7febdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:31.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.010+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7febe40efc30 con 0x7febf8077040 2024-09-12T03:25:31.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.010+0000 7febf7e00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7febdc0a2c10 0x7febdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:31.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.011+0000 7febf7e00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7febdc0a2c10 0x7febdc0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febf819fa20 tx=0x7febe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:31.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.014+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febe40bc020 con 0x7febf8077040 2024-09-12T03:25:31.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.231+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7febf810dc20 con 0x7febdc0a2c10 2024-09-12T03:25:31.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.236+0000 7febf4c00640 1 -- 172.21.15.50:0/837673349 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7febf810dc20 con 0x7febdc0a2c10 2024-09-12T03:25:31.236 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/qa/sv_0/087d2660-6352-453f-92b4-a94d73097f66 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.238+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7febdc0a2c10 msgr2=0x7febdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.238+0000 7febfea00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7febdc0a2c10 0x7febdc0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febf819fa20 tx=0x7febe8005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.238+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 msgr2=0x7febf81a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.238+0000 7febfea00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81a4c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7febe4004770 tx=0x7febe4003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 shutdown_connections 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7febdc0a2c10 0x7febdc0a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf8112a30 0x7febf819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febf8077040 0x7febf81a4c60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 --2- 172.21.15.50:0/837673349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf8076720 0x7febf81a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:31.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 >> 172.21.15.50:0/837673349 conn(0x7febf80ffa90 msgr2=0x7febf8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:31.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 shutdown_connections 2024-09-12T03:25:31.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:31.239+0000 7febfea00640 1 -- 172.21.15.50:0/837673349 wait complete. 2024-09-12T03:25:31.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:31 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:31.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:31 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:31.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:31 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:31.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:31 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:31.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:31 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:31.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:31 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:31.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:31 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:31.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:31 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:31.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:31 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:25:31.950 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T03:25:32.328 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:32.751 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:32 smithi050 ceph-mon[30378]: pgmap v331: 129 pgs: 129 active+clean; 604 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:25:32.751 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:32 smithi050 ceph-mon[30378]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:32 smithi082 ceph-mon[35046]: pgmap v331: 129 pgs: 129 active+clean; 604 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:25:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:32 smithi082 ceph-mon[35046]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:32.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 -- 172.21.15.50:0/2459003731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c103ff0 msgr2=0x7fc58c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:32.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 --2- 172.21.15.50:0/2459003731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c103ff0 0x7fc58c1043d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc580009900 tx=0x7fc58002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:32.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 -- 172.21.15.50:0/2459003731 shutdown_connections 2024-09-12T03:25:32.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 --2- 172.21.15.50:0/2459003731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c109120 0x7fc58c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:32.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 --2- 172.21.15.50:0/2459003731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc58c1049a0 0x7fc58c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:32.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 --2- 172.21.15.50:0/2459003731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c103ff0 0x7fc58c1043d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:32.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 -- 172.21.15.50:0/2459003731 >> 172.21.15.50:0/2459003731 conn(0x7fc58c0ffa90 msgr2=0x7fc58c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:32.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 -- 172.21.15.50:0/2459003731 shutdown_connections 2024-09-12T03:25:32.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.869+0000 7fc594200640 1 -- 172.21.15.50:0/2459003731 wait complete. 2024-09-12T03:25:32.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.870+0000 7fc594200640 1 Processor -- start 2024-09-12T03:25:32.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.870+0000 7fc594200640 1 -- start start 2024-09-12T03:25:32.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc594200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c103ff0 0x7fc58c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:32.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc594200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc58c1049a0 0x7fc58c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:32.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc594200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 0x7fc58c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc594200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c114120 con 0x7fc58c109120 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc594200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c113fa0 con 0x7fc58c103ff0 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc594200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c1142a0 con 0x7fc58c1049a0 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 0x7fc58c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 0x7fc58c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44796/0 (socket says 172.21.15.50:44796) 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 -- 172.21.15.50:0/3349168979 learned_addr learned my addr 172.21.15.50:0/3349168979 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc591800640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c103ff0 0x7fc58c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc590e00640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc58c1049a0 0x7fc58c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 -- 172.21.15.50:0/3349168979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc58c1049a0 msgr2=0x7fc58c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc58c1049a0 0x7fc58c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 -- 172.21.15.50:0/3349168979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c103ff0 msgr2=0x7fc58c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c103ff0 0x7fc58c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:32.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.871+0000 7fc592200640 1 -- 172.21.15.50:0/3349168979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c19f590 con 0x7fc58c109120 2024-09-12T03:25:32.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.872+0000 7fc592200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 0x7fc58c19eeb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc58400eb10 tx=0x7fc58400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:32.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.872+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584004b20 con 0x7fc58c109120 2024-09-12T03:25:32.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.872+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc584004cc0 con 0x7fc58c109120 2024-09-12T03:25:32.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.872+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c19f880 con 0x7fc58c109120 2024-09-12T03:25:32.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.872+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584010760 con 0x7fc58c109120 2024-09-12T03:25:32.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.872+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c1a7160 con 0x7fc58c109120 2024-09-12T03:25:32.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.873+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c1043d0 con 0x7fc58c109120 2024-09-12T03:25:32.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.875+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc584005020 con 0x7fc58c109120 2024-09-12T03:25:32.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.876+0000 7fc58a000640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5740a2c10 0x7fc5740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:32.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.877+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc5840cfa80 con 0x7fc58c109120 2024-09-12T03:25:32.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.877+0000 7fc591800640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5740a2c10 0x7fc5740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:32.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.878+0000 7fc591800640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5740a2c10 0x7fc5740a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc580009230 tx=0x7fc5800058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:32.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:32.881+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5840d1e20 con 0x7fc58c109120 2024-09-12T03:25:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:32 smithi142 ceph-mon[35329]: pgmap v331: 129 pgs: 129 active+clean; 604 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:25:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:32 smithi142 ceph-mon[35329]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:33.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.095+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc58c062630 con 0x7fc5740a2c10 2024-09-12T03:25:33.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.122+0000 7fc58a000640 1 -- 172.21.15.50:0/3349168979 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc58c062630 con 0x7fc5740a2c10 2024-09-12T03:25:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5740a2c10 msgr2=0x7fc5740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5740a2c10 0x7fc5740a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc580009230 tx=0x7fc5800058d0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 msgr2=0x7fc58c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 0x7fc58c19eeb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc58400eb10 tx=0x7fc58400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 shutdown_connections 2024-09-12T03:25:33.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5740a2c10 0x7fc5740a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc58c109120 0x7fc58c19eeb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc58c1049a0 0x7fc58c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 --2- 172.21.15.50:0/3349168979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c103ff0 0x7fc58c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.124+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 >> 172.21.15.50:0/3349168979 conn(0x7fc58c0ffa90 msgr2=0x7fc58c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.125+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 shutdown_connections 2024-09-12T03:25:33.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:33.125+0000 7fc594200640 1 -- 172.21.15.50:0/3349168979 wait complete. 2024-09-12T03:25:33.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T03:25:34.164 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 -- 172.21.15.50:0/153038897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c103e30 msgr2=0x7ff83c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 --2- 172.21.15.50:0/153038897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c103e30 0x7ff83c104210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009900 tx=0x7ff82c02a410 comp rx=0 tx=0).stop 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 -- 172.21.15.50:0/153038897 shutdown_connections 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 --2- 172.21.15.50:0/153038897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff83c108f60 0x7ff83c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 --2- 172.21.15.50:0/153038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c1047e0 0x7ff83c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 --2- 172.21.15.50:0/153038897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c103e30 0x7ff83c104210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 -- 172.21.15.50:0/153038897 >> 172.21.15.50:0/153038897 conn(0x7ff83c0ff8d0 msgr2=0x7ff83c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 -- 172.21.15.50:0/153038897 shutdown_connections 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.698+0000 7ff841600640 1 -- 172.21.15.50:0/153038897 wait complete. 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 Processor -- start 2024-09-12T03:25:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 -- start start 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff83c103e30 0x7ff83c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c1047e0 0x7ff83c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 0x7ff83c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c114050 con 0x7ff83c103e30 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c113ed0 con 0x7ff83c108f60 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.699+0000 7ff841600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1141d0 con 0x7ff83c1047e0 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 0x7ff83c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 0x7ff83c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41010/0 (socket says 172.21.15.50:41010) 2024-09-12T03:25:34.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 -- 172.21.15.50:0/2835649469 learned_addr learned my addr 172.21.15.50:0/2835649469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff83c103e30 0x7ff83c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff833e00640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c1047e0 0x7ff83c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 -- 172.21.15.50:0/2835649469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c1047e0 msgr2=0x7ff83c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c1047e0 0x7ff83c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 -- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff83c103e30 msgr2=0x7ff83c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff83c103e30 0x7ff83c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.700+0000 7ff840c00640 1 -- 172.21.15.50:0/2835649469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c19f4e0 con 0x7ff83c108f60 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.701+0000 7ff840c00640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 0x7ff83c19ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff83400eb10 tx=0x7ff83400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.701+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834004b20 con 0x7ff83c108f60 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.701+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff834004cc0 con 0x7ff83c108f60 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.701+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834010760 con 0x7ff83c108f60 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.701+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c19f7d0 con 0x7ff83c108f60 2024-09-12T03:25:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.701+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c1a70b0 con 0x7ff83c108f60 2024-09-12T03:25:34.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.702+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c104210 con 0x7ff83c108f60 2024-09-12T03:25:34.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.704+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff834005000 con 0x7ff83c108f60 2024-09-12T03:25:34.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.706+0000 7ff839600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8200a2c90 0x7ff8200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:34.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.706+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff8340cfcb0 con 0x7ff83c108f60 2024-09-12T03:25:34.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.706+0000 7ff83be00640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8200a2c90 0x7ff8200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:34.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.706+0000 7ff83be00640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8200a2c90 0x7ff8200a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000c00 tx=0x7ff82c005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:34.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.708+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8340541b0 con 0x7ff83c108f60 2024-09-12T03:25:34.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:34 smithi082 ceph-mon[35046]: pgmap v332: 129 pgs: 129 active+clean; 604 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:25:34.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:34 smithi082 ceph-mon[35046]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:34 smithi050 ceph-mon[30378]: pgmap v332: 129 pgs: 129 active+clean; 604 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:25:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:34 smithi050 ceph-mon[30378]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:34.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.922+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff83c062630 con 0x7ff8200a2c90 2024-09-12T03:25:34.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.927+0000 7ff839600640 1 -- 172.21.15.50:0/2835649469 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff83c062630 con 0x7ff8200a2c90 2024-09-12T03:25:34.926 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/qa/sv_1/d81a0432-0cba-4da2-80bc-4beb3a972d69 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8200a2c90 msgr2=0x7ff8200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8200a2c90 0x7ff8200a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000c00 tx=0x7ff82c005b50 comp rx=0 tx=0).stop 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 msgr2=0x7ff83c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 0x7ff83c19ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff83400eb10 tx=0x7ff83400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 shutdown_connections 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff8200a2c90 0x7ff8200a5150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c108f60 0x7ff83c19ee00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c1047e0 0x7ff83c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 --2- 172.21.15.50:0/2835649469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff83c103e30 0x7ff83c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:34.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 >> 172.21.15.50:0/2835649469 conn(0x7ff83c0ff8d0 msgr2=0x7ff83c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:34.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 shutdown_connections 2024-09-12T03:25:34.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:34.929+0000 7ff841600640 1 -- 172.21.15.50:0/2835649469 wait complete. 2024-09-12T03:25:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:34 smithi142 ceph-mon[35329]: pgmap v332: 129 pgs: 129 active+clean; 604 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:25:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:34 smithi142 ceph-mon[35329]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:35.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:35.995 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:36.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 -- 172.21.15.50:0/2160275231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 msgr2=0x7f56f4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 --2- 172.21.15.50:0/2160275231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f4074a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009900 tx=0x7f56e802a480 comp rx=0 tx=0).stop 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 -- 172.21.15.50:0/2160275231 shutdown_connections 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 --2- 172.21.15.50:0/2160275231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f410e3c0 0x7f56f4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 --2- 172.21.15.50:0/2160275231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f4074f40 0x7f56f410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 --2- 172.21.15.50:0/2160275231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f4074a00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 -- 172.21.15.50:0/2160275231 >> 172.21.15.50:0/2160275231 conn(0x7f56f40fd8c0 msgr2=0x7f56f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 -- 172.21.15.50:0/2160275231 shutdown_connections 2024-09-12T03:25:36.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.571+0000 7f56fc800640 1 -- 172.21.15.50:0/2160275231 wait complete. 2024-09-12T03:25:36.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.572+0000 7f56fc800640 1 Processor -- start 2024-09-12T03:25:36.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.572+0000 7f56fc800640 1 -- start start 2024-09-12T03:25:36.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fc800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fc800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 0x7f56f419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fc800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f410e3c0 0x7f56f419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fc800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41141c0 con 0x7f56f4074620 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fc800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114040 con 0x7f56f410e3c0 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fc800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114340 con 0x7f56f4074f40 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 0x7f56f419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 0x7f56f419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58862/0 (socket says 172.21.15.50:58862) 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44828/0 (socket says 172.21.15.50:44828) 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 -- 172.21.15.50:0/1886128768 learned_addr learned my addr 172.21.15.50:0/1886128768 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56fa800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f410e3c0 0x7f56f419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 -- 172.21.15.50:0/1886128768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f410e3c0 msgr2=0x7f56f419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f410e3c0 0x7f56f419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 -- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 msgr2=0x7f56f419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.573+0000 7f56f9400640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56f9400640 1 -- 172.21.15.50:0/1886128768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f419f640 con 0x7f56f4074f40 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56fa800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f410e3c0 0x7f56f419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56f9400640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 0x7f56f419abd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56e400e9f0 tx=0x7f56e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e400cd80 con 0x7f56f4074f40 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e4010910 con 0x7f56f4074f40 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f419f930 con 0x7f56f4074f40 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4018990 con 0x7f56f4074f40 2024-09-12T03:25:36.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.574+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f41a7210 con 0x7f56f4074f40 2024-09-12T03:25:36.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.576+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f4074a00 con 0x7f56f4074f40 2024-09-12T03:25:36.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.577+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f56e4010ac0 con 0x7f56f4074f40 2024-09-12T03:25:36.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.578+0000 7f56eea00640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56dc0a2c90 0x7f56dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.579+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f56e40cf8b0 con 0x7f56f4074f40 2024-09-12T03:25:36.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.579+0000 7f56f9e00640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56dc0a2c90 0x7f56dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.579+0000 7f56f9e00640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56dc0a2c90 0x7f56dc0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56e8000c00 tx=0x7f56e8005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.583+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56e409c2f0 con 0x7f56f4074f40 2024-09-12T03:25:36.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.809+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56f402d7c0 con 0x7f56f4074f40 2024-09-12T03:25:36.810 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:36 smithi050 ceph-mon[30378]: pgmap v333: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-09-12T03:25:36.810 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:36 smithi050 ceph-mon[30378]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:36.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.810+0000 7f56eea00640 1 -- 172.21.15.50:0/1886128768 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f56e409bbf0 con 0x7f56f4074f40 2024-09-12T03:25:36.811 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"btime":"2024-09-12T03:25:24:859723+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:25:24.725697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24508,24526,24536]},"id":1}]} 2024-09-12T03:25:36.812 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:25:36.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.813+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56dc0a2c90 msgr2=0x7f56dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.813+0000 7f56fc800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56dc0a2c90 0x7f56dc0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56e8000c00 tx=0x7f56e8005b50 comp rx=0 tx=0).stop 2024-09-12T03:25:36.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.813+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 msgr2=0x7f56f419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.813+0000 7f56fc800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 0x7f56f419abd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56e400e9f0 tx=0x7f56e400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 shutdown_connections 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f56dc0a2c90 0x7f56dc0a5150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f410e3c0 0x7f56f419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f4074f40 0x7f56f419abd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 --2- 172.21.15.50:0/1886128768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56f4074620 0x7f56f419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 >> 172.21.15.50:0/1886128768 conn(0x7f56f40fd8c0 msgr2=0x7f56f40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 shutdown_connections 2024-09-12T03:25:36.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:36.814+0000 7f56fc800640 1 -- 172.21.15.50:0/1886128768 wait complete. 2024-09-12T03:25:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:36 smithi082 ceph-mon[35046]: pgmap v333: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-09-12T03:25:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:36 smithi082 ceph-mon[35046]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:36 smithi142 ceph-mon[35329]: pgmap v333: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-09-12T03:25:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:36 smithi142 ceph-mon[35329]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:37.586 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:25:37.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:37 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1886128768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:37.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:37 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1886128768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:37.970 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:38.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:37 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1886128768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 -- 172.21.15.50:0/2555495306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074510 msgr2=0x7fceb40748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 --2- 172.21.15.50:0/2555495306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074510 0x7fceb40748f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009900 tx=0x7fcea802a480 comp rx=0 tx=0).stop 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 -- 172.21.15.50:0/2555495306 shutdown_connections 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 --2- 172.21.15.50:0/2555495306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb410e280 0x7fceb4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 --2- 172.21.15.50:0/2555495306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb4074e30 0x7fceb410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 --2- 172.21.15.50:0/2555495306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074510 0x7fceb40748f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 -- 172.21.15.50:0/2555495306 >> 172.21.15.50:0/2555495306 conn(0x7fceb40fd740 msgr2=0x7fceb40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 -- 172.21.15.50:0/2555495306 shutdown_connections 2024-09-12T03:25:38.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.471+0000 7fceba200640 1 -- 172.21.15.50:0/2555495306 wait complete. 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 Processor -- start 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 -- start start 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4074510 0x7fceb419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 0x7fceb419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb410e280 0x7fceb419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4113fb0 con 0x7fceb4074e30 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4113e30 con 0x7fceb4074510 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.472+0000 7fceba200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4114130 con 0x7fceb410e280 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 0x7fceb419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4074510 0x7fceb419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 0x7fceb419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44844/0 (socket says 172.21.15.50:44844) 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb410e280 0x7fceb419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb3e00640 1 -- 172.21.15.50:0/1602809823 learned_addr learned my addr 172.21.15.50:0/1602809823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb8e00640 1 -- 172.21.15.50:0/1602809823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb410e280 msgr2=0x7fceb419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb8e00640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb410e280 0x7fceb419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb8e00640 1 -- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 msgr2=0x7fceb419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb8e00640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 0x7fceb419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb8e00640 1 -- 172.21.15.50:0/1602809823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb419f500 con 0x7fceb4074510 2024-09-12T03:25:38.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.473+0000 7fceb3e00640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 0x7fceb419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:38.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.474+0000 7fceb8e00640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4074510 0x7fceb419a550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcea8005940 tx=0x7fcea80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.474+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea803f030 con 0x7fceb4074510 2024-09-12T03:25:38.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.474+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb419f790 con 0x7fceb4074510 2024-09-12T03:25:38.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.474+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea8003ce0 con 0x7fceb4074510 2024-09-12T03:25:38.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.474+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8035070 con 0x7fceb4074510 2024-09-12T03:25:38.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.474+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41a70d0 con 0x7fceb4074510 2024-09-12T03:25:38.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.475+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb40748f0 con 0x7fceb4074510 2024-09-12T03:25:38.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.477+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcea8042030 con 0x7fceb4074510 2024-09-12T03:25:38.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.479+0000 7fceb1600640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2c90 0x7fce9c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:38.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.479+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcea80eff50 con 0x7fceb4074510 2024-09-12T03:25:38.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.479+0000 7fceb3e00640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2c90 0x7fce9c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:38.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.480+0000 7fceb3e00640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2c90 0x7fce9c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fceb40ffd60 tx=0x7fcea000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:38.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.481+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcea80bc340 con 0x7fceb4074510 2024-09-12T03:25:38.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.671+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fceb4109400 con 0x7fceb4074510 2024-09-12T03:25:38.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.673+0000 7fceb1600640 1 -- 172.21.15.50:0/1602809823 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24622 (secure 0 0 0) 0x7fcea80c0640 con 0x7fceb4074510 2024-09-12T03:25:38.673 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:38.674 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":92,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:25:13.804593+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:25:38.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.675+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2c90 msgr2=0x7fce9c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.675+0000 7fceba200640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2c90 0x7fce9c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fceb40ffd60 tx=0x7fcea000b410 comp rx=0 tx=0).stop 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.675+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4074510 msgr2=0x7fceb419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.675+0000 7fceba200640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4074510 0x7fceb419a550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcea8005940 tx=0x7fcea80042f0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 shutdown_connections 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fce9c0a2c90 0x7fce9c0a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceb410e280 0x7fceb419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceb4074e30 0x7fceb419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 --2- 172.21.15.50:0/1602809823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4074510 0x7fceb419a550 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 >> 172.21.15.50:0/1602809823 conn(0x7fceb40fd740 msgr2=0x7fceb40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 shutdown_connections 2024-09-12T03:25:38.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:38.676+0000 7fceba200640 1 -- 172.21.15.50:0/1602809823 wait complete. 2024-09-12T03:25:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:38 smithi082 ceph-mon[35046]: pgmap v334: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-12T03:25:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:38 smithi050 ceph-mon[30378]: pgmap v334: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-12T03:25:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:38 smithi142 ceph-mon[35329]: pgmap v334: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-12T03:25:39.415 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:25:39.416 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:25:39.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:39 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1602809823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:39.797 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:39.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:39 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1602809823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:40.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:39 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1602809823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:40.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 -- 172.21.15.50:0/2474124950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 msgr2=0x7f1810076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 --2- 172.21.15.50:0/2474124950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f1810076b00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1800009860 tx=0x7f180002a340 comp rx=0 tx=0).stop 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 -- 172.21.15.50:0/2474124950 shutdown_connections 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 --2- 172.21.15.50:0/2474124950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1810112a30 0x7f1810114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 --2- 172.21.15.50:0/2474124950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1810077040 0x7f18101124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 --2- 172.21.15.50:0/2474124950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f1810076b00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.293+0000 7f1815000640 1 -- 172.21.15.50:0/2474124950 >> 172.21.15.50:0/2474124950 conn(0x7f18100ffa90 msgr2=0x7f1810101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.294+0000 7f1815000640 1 -- 172.21.15.50:0/2474124950 shutdown_connections 2024-09-12T03:25:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.294+0000 7f1815000640 1 -- 172.21.15.50:0/2474124950 wait complete. 2024-09-12T03:25:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.294+0000 7f1815000640 1 Processor -- start 2024-09-12T03:25:40.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 -- start start 2024-09-12T03:25:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f18101a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1810077040 0x7f18101a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1810112a30 0x7f181019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f18101185c0 con 0x7f1810112a30 2024-09-12T03:25:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1810118440 con 0x7f1810077040 2024-09-12T03:25:40.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1815000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1810118740 con 0x7f1810076720 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f180de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f18101a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f180de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f18101a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:46146/0 (socket says 172.21.15.50:46146) 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f180de00640 1 -- 172.21.15.50:0/1516731664 learned_addr learned my addr 172.21.15.50:0/1516731664 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f180e800640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1810112a30 0x7f181019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.295+0000 7f1807e00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1810077040 0x7f18101a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f180de00640 1 -- 172.21.15.50:0/1516731664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1810077040 msgr2=0x7f18101a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f180de00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1810077040 0x7f18101a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f180de00640 1 -- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1810112a30 msgr2=0x7f181019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f180de00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1810112a30 0x7f181019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f180de00640 1 -- 172.21.15.50:0/1516731664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181019efd0 con 0x7f1810076720 2024-09-12T03:25:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f180de00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f18101a46e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1800035040 tx=0x7f1800033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180003f020 con 0x7f1810076720 2024-09-12T03:25:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1800032770 con 0x7f1810076720 2024-09-12T03:25:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181019f260 con 0x7f1810076720 2024-09-12T03:25:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.296+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800037040 con 0x7f1810076720 2024-09-12T03:25:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.297+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101ab5b0 con 0x7f1810076720 2024-09-12T03:25:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.298+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1810076b00 con 0x7f1810076720 2024-09-12T03:25:40.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.300+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1800042020 con 0x7f1810076720 2024-09-12T03:25:40.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.301+0000 7f1806a00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f17f40a2c10 0x7f17f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.301+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f18000ef9d0 con 0x7f1810076720 2024-09-12T03:25:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.301+0000 7f1807e00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f17f40a2c10 0x7f17f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.302+0000 7f1807e00640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f17f40a2c10 0x7f17f40a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1810079300 tx=0x7f17f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.303+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1800049020 con 0x7f1810076720 2024-09-12T03:25:40.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.486+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18101146e0 con 0x7f17f40a2c10 2024-09-12T03:25:40.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.497+0000 7f1806a00640 1 -- 172.21.15.50:0/1516731664 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340354 (secure 0 0 0) 0x7f18101146e0 con 0x7f17f40a2c10 2024-09-12T03:25:40.502 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:40.504 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f17f40a2c10 msgr2=0x7f17f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f17f40a2c10 0x7f17f40a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1810079300 tx=0x7f17f8009260 comp rx=0 tx=0).stop 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 msgr2=0x7f18101a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f18101a46e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1800035040 tx=0x7f1800033d90 comp rx=0 tx=0).stop 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 shutdown_connections 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f17f40a2c10 0x7f17f40a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1810112a30 0x7f181019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1810077040 0x7f18101a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 --2- 172.21.15.50:0/1516731664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1810076720 0x7f18101a46e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 >> 172.21.15.50:0/1516731664 conn(0x7f18100ffa90 msgr2=0x7f1810101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.501+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 shutdown_connections 2024-09-12T03:25:40.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:40.502+0000 7f1815000640 1 -- 172.21.15.50:0/1516731664 wait complete. 2024-09-12T03:25:40.837 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:40 smithi050 ceph-mon[30378]: pgmap v335: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:25:40.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:40 smithi082 ceph-mon[35046]: pgmap v335: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:25:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:40 smithi142 ceph-mon[35329]: pgmap v335: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:25:41.167 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-12T03:25:38.761042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637950,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":339,"num_write_kb":3033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376016,"kb_used_data":14288,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565808,"statfs":{"total":1151940427776,"available":1151555387392,"internally_reserved":0,"allocated":14630912,"data_stored":8139223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40128,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":76,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003345"},"pg_stats":[{"pgid":"2.1f","version":"92'9","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.863741+0000","last_change":"2024-09-12T03:24:29.302864+0000","last_active":"2024-09-12T03:25:21.863741+0000","last_peered":"2024-09-12T03:25:21.863741+0000","last_clean":"2024-09-12T03:25:21.863741+0000","last_became_active":"2024-09-12T03:24:29.302551+0000","last_became_peered":"2024-09-12T03:24:29.302551+0000","last_unstale":"2024-09-12T03:25:21.863741+0000","last_undegraded":"2024-09-12T03:25:21.863741+0000","last_fullsized":"2024-09-12T03:25:21.863741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:10.151772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.887020+0000","last_change":"2024-09-12T03:24:31.649557+0000","last_active":"2024-09-12T03:25:13.887020+0000","last_peered":"2024-09-12T03:25:13.887020+0000","last_clean":"2024-09-12T03:25:13.887020+0000","last_became_active":"2024-09-12T03:24:31.649082+0000","last_became_peered":"2024-09-12T03:24:31.649082+0000","last_unstale":"2024-09-12T03:25:13.887020+0000","last_undegraded":"2024-09-12T03:25:13.887020+0000","last_fullsized":"2024-09-12T03:25:13.887020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:54.245063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810710+0000","last_change":"2024-09-12T03:24:40.708670+0000","last_active":"2024-09-12T03:25:13.810710+0000","last_peered":"2024-09-12T03:25:13.810710+0000","last_clean":"2024-09-12T03:25:13.810710+0000","last_became_active":"2024-09-12T03:24:40.708264+0000","last_became_peered":"2024-09-12T03:24:40.708264+0000","last_unstale":"2024-09-12T03:25:13.810710+0000","last_undegraded":"2024-09-12T03:25:13.810710+0000","last_fullsized":"2024-09-12T03:25:13.810710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:00.670492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.887210+0000","last_change":"2024-09-12T03:24:29.312254+0000","last_active":"2024-09-12T03:25:13.887210+0000","last_peered":"2024-09-12T03:25:13.887210+0000","last_clean":"2024-09-12T03:25:13.887210+0000","last_became_active":"2024-09-12T03:24:29.311842+0000","last_became_peered":"2024-09-12T03:24:29.311842+0000","last_unstale":"2024-09-12T03:25:13.887210+0000","last_undegraded":"2024-09-12T03:25:13.887210+0000","last_fullsized":"2024-09-12T03:25:13.887210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:43.298456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811346+0000","last_change":"2024-09-12T03:24:31.644330+0000","last_active":"2024-09-12T03:25:13.811346+0000","last_peered":"2024-09-12T03:25:13.811346+0000","last_clean":"2024-09-12T03:25:13.811346+0000","last_became_active":"2024-09-12T03:24:31.642904+0000","last_became_peered":"2024-09-12T03:24:31.642904+0000","last_unstale":"2024-09-12T03:25:13.811346+0000","last_undegraded":"2024-09-12T03:25:13.811346+0000","last_fullsized":"2024-09-12T03:25:13.811346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:14.304333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812283+0000","last_change":"2024-09-12T03:24:40.706682+0000","last_active":"2024-09-12T03:25:13.812283+0000","last_peered":"2024-09-12T03:25:13.812283+0000","last_clean":"2024-09-12T03:25:13.812283+0000","last_became_active":"2024-09-12T03:24:40.706143+0000","last_became_peered":"2024-09-12T03:24:40.706143+0000","last_unstale":"2024-09-12T03:25:13.812283+0000","last_undegraded":"2024-09-12T03:25:13.812283+0000","last_fullsized":"2024-09-12T03:25:13.812283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:05.146486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.862104+0000","last_change":"2024-09-12T03:24:29.307621+0000","last_active":"2024-09-12T03:25:21.862104+0000","last_peered":"2024-09-12T03:25:21.862104+0000","last_clean":"2024-09-12T03:25:21.862104+0000","last_became_active":"2024-09-12T03:24:29.307258+0000","last_became_peered":"2024-09-12T03:24:29.307258+0000","last_unstale":"2024-09-12T03:25:21.862104+0000","last_undegraded":"2024-09-12T03:25:21.862104+0000","last_fullsized":"2024-09-12T03:25:21.862104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:03.513367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815361+0000","last_change":"2024-09-12T03:24:31.649972+0000","last_active":"2024-09-12T03:25:13.815361+0000","last_peered":"2024-09-12T03:25:13.815361+0000","last_clean":"2024-09-12T03:25:13.815361+0000","last_became_active":"2024-09-12T03:24:31.649693+0000","last_became_peered":"2024-09-12T03:24:31.649693+0000","last_unstale":"2024-09-12T03:25:13.815361+0000","last_undegraded":"2024-09-12T03:25:13.815361+0000","last_fullsized":"2024-09-12T03:25:13.815361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:10.502054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813448+0000","last_change":"2024-09-12T03:24:40.698995+0000","last_active":"2024-09-12T03:25:13.813448+0000","last_peered":"2024-09-12T03:25:13.813448+0000","last_clean":"2024-09-12T03:25:13.813448+0000","last_became_active":"2024-09-12T03:24:40.698094+0000","last_became_peered":"2024-09-12T03:24:40.698094+0000","last_unstale":"2024-09-12T03:25:13.813448+0000","last_undegraded":"2024-09-12T03:25:13.813448+0000","last_fullsized":"2024-09-12T03:25:13.813448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:39.266209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809719+0000","last_change":"2024-09-12T03:24:29.304522+0000","last_active":"2024-09-12T03:25:13.809719+0000","last_peered":"2024-09-12T03:25:13.809719+0000","last_clean":"2024-09-12T03:25:13.809719+0000","last_became_active":"2024-09-12T03:24:29.304206+0000","last_became_peered":"2024-09-12T03:24:29.304206+0000","last_unstale":"2024-09-12T03:25:13.809719+0000","last_undegraded":"2024-09-12T03:25:13.809719+0000","last_fullsized":"2024-09-12T03:25:13.809719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:12.365720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320244+0000","last_change":"2024-09-12T03:24:31.648735+0000","last_active":"2024-09-12T03:25:14.320244+0000","last_peered":"2024-09-12T03:25:14.320244+0000","last_clean":"2024-09-12T03:25:14.320244+0000","last_became_active":"2024-09-12T03:24:31.648416+0000","last_became_peered":"2024-09-12T03:24:31.648416+0000","last_unstale":"2024-09-12T03:25:14.320244+0000","last_undegraded":"2024-09-12T03:25:14.320244+0000","last_fullsized":"2024-09-12T03:25:14.320244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:24.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813389+0000","last_change":"2024-09-12T03:24:40.699941+0000","last_active":"2024-09-12T03:25:13.813389+0000","last_peered":"2024-09-12T03:25:13.813389+0000","last_clean":"2024-09-12T03:25:13.813389+0000","last_became_active":"2024-09-12T03:24:40.699126+0000","last_became_peered":"2024-09-12T03:24:40.699126+0000","last_unstale":"2024-09-12T03:25:13.813389+0000","last_undegraded":"2024-09-12T03:25:13.813389+0000","last_fullsized":"2024-09-12T03:25:13.813389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:24.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788924+0000","last_change":"2024-09-12T03:24:31.650650+0000","last_active":"2024-09-12T03:25:14.788924+0000","last_peered":"2024-09-12T03:25:14.788924+0000","last_clean":"2024-09-12T03:25:14.788924+0000","last_became_active":"2024-09-12T03:24:31.650279+0000","last_became_peered":"2024-09-12T03:24:31.650279+0000","last_unstale":"2024-09-12T03:25:14.788924+0000","last_undegraded":"2024-09-12T03:25:14.788924+0000","last_fullsized":"2024-09-12T03:25:14.788924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:49.860317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857601+0000","last_change":"2024-09-12T03:24:29.307426+0000","last_active":"2024-09-12T03:25:20.857601+0000","last_peered":"2024-09-12T03:25:20.857601+0000","last_clean":"2024-09-12T03:25:20.857601+0000","last_became_active":"2024-09-12T03:24:29.307119+0000","last_became_peered":"2024-09-12T03:24:29.307119+0000","last_unstale":"2024-09-12T03:25:20.857601+0000","last_undegraded":"2024-09-12T03:25:20.857601+0000","last_fullsized":"2024-09-12T03:25:20.857601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:48.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812724+0000","last_change":"2024-09-12T03:24:40.700292+0000","last_active":"2024-09-12T03:25:13.812724+0000","last_peered":"2024-09-12T03:25:13.812724+0000","last_clean":"2024-09-12T03:25:13.812724+0000","last_became_active":"2024-09-12T03:24:40.699819+0000","last_became_peered":"2024-09-12T03:24:40.699819+0000","last_unstale":"2024-09-12T03:25:13.812724+0000","last_undegraded":"2024-09-12T03:25:13.812724+0000","last_fullsized":"2024-09-12T03:25:13.812724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:43.022322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886544+0000","last_change":"2024-09-12T03:24:31.647669+0000","last_active":"2024-09-12T03:25:13.886544+0000","last_peered":"2024-09-12T03:25:13.886544+0000","last_clean":"2024-09-12T03:25:13.886544+0000","last_became_active":"2024-09-12T03:24:31.644429+0000","last_became_peered":"2024-09-12T03:24:31.644429+0000","last_unstale":"2024-09-12T03:25:13.886544+0000","last_undegraded":"2024-09-12T03:25:13.886544+0000","last_fullsized":"2024-09-12T03:25:13.886544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:22.556399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811172+0000","last_change":"2024-09-12T03:24:29.309334+0000","last_active":"2024-09-12T03:25:13.811172+0000","last_peered":"2024-09-12T03:25:13.811172+0000","last_clean":"2024-09-12T03:25:13.811172+0000","last_became_active":"2024-09-12T03:24:29.309007+0000","last_became_peered":"2024-09-12T03:24:29.309007+0000","last_unstale":"2024-09-12T03:25:13.811172+0000","last_undegraded":"2024-09-12T03:25:13.811172+0000","last_fullsized":"2024-09-12T03:25:13.811172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:40.898104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812855+0000","last_change":"2024-09-12T03:24:40.706611+0000","last_active":"2024-09-12T03:25:13.812855+0000","last_peered":"2024-09-12T03:25:13.812855+0000","last_clean":"2024-09-12T03:25:13.812855+0000","last_became_active":"2024-09-12T03:24:40.706058+0000","last_became_peered":"2024-09-12T03:24:40.706058+0000","last_unstale":"2024-09-12T03:25:13.812855+0000","last_undegraded":"2024-09-12T03:25:13.812855+0000","last_fullsized":"2024-09-12T03:25:13.812855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:16.547894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265106+0000","last_change":"2024-09-12T03:24:31.648458+0000","last_active":"2024-09-12T03:25:14.265106+0000","last_peered":"2024-09-12T03:25:14.265106+0000","last_clean":"2024-09-12T03:25:14.265106+0000","last_became_active":"2024-09-12T03:24:31.644171+0000","last_became_peered":"2024-09-12T03:24:31.644171+0000","last_unstale":"2024-09-12T03:25:14.265106+0000","last_undegraded":"2024-09-12T03:25:14.265106+0000","last_fullsized":"2024-09-12T03:25:14.265106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:20.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.858013+0000","last_change":"2024-09-12T03:24:29.309013+0000","last_active":"2024-09-12T03:25:20.858013+0000","last_peered":"2024-09-12T03:25:20.858013+0000","last_clean":"2024-09-12T03:25:20.858013+0000","last_became_active":"2024-09-12T03:24:29.308459+0000","last_became_peered":"2024-09-12T03:24:29.308459+0000","last_unstale":"2024-09-12T03:25:20.858013+0000","last_undegraded":"2024-09-12T03:25:20.858013+0000","last_fullsized":"2024-09-12T03:25:20.858013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:38.655615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.321811+0000","last_change":"2024-09-12T03:25:14.321079+0000","last_active":"2024-09-12T03:25:14.321811+0000","last_peered":"2024-09-12T03:25:14.321811+0000","last_clean":"2024-09-12T03:25:14.321811+0000","last_became_active":"2024-09-12T03:25:14.320602+0000","last_became_peered":"2024-09-12T03:25:14.320602+0000","last_unstale":"2024-09-12T03:25:14.321811+0000","last_undegraded":"2024-09-12T03:25:14.321811+0000","last_fullsized":"2024-09-12T03:25:14.321811+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:18.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809174+0000","last_change":"2024-09-12T03:24:31.637560+0000","last_active":"2024-09-12T03:25:13.809174+0000","last_peered":"2024-09-12T03:25:13.809174+0000","last_clean":"2024-09-12T03:25:13.809174+0000","last_became_active":"2024-09-12T03:24:31.636962+0000","last_became_peered":"2024-09-12T03:24:31.636962+0000","last_unstale":"2024-09-12T03:25:13.809174+0000","last_undegraded":"2024-09-12T03:25:13.809174+0000","last_fullsized":"2024-09-12T03:25:13.809174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:44.468397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857965+0000","last_change":"2024-09-12T03:24:29.305262+0000","last_active":"2024-09-12T03:25:20.857965+0000","last_peered":"2024-09-12T03:25:20.857965+0000","last_clean":"2024-09-12T03:25:20.857965+0000","last_became_active":"2024-09-12T03:24:29.304937+0000","last_became_peered":"2024-09-12T03:24:29.304937+0000","last_unstale":"2024-09-12T03:25:20.857965+0000","last_undegraded":"2024-09-12T03:25:20.857965+0000","last_fullsized":"2024-09-12T03:25:20.857965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:45.172526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812951+0000","last_change":"2024-09-12T03:24:40.702517+0000","last_active":"2024-09-12T03:25:13.812951+0000","last_peered":"2024-09-12T03:25:13.812951+0000","last_clean":"2024-09-12T03:25:13.812951+0000","last_became_active":"2024-09-12T03:24:40.702093+0000","last_became_peered":"2024-09-12T03:24:40.702093+0000","last_unstale":"2024-09-12T03:25:13.812951+0000","last_undegraded":"2024-09-12T03:25:13.812951+0000","last_fullsized":"2024-09-12T03:25:13.812951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:00.817214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811275+0000","last_change":"2024-09-12T03:24:31.650541+0000","last_active":"2024-09-12T03:25:13.811275+0000","last_peered":"2024-09-12T03:25:13.811275+0000","last_clean":"2024-09-12T03:25:13.811275+0000","last_became_active":"2024-09-12T03:24:31.650180+0000","last_became_peered":"2024-09-12T03:24:31.650180+0000","last_unstale":"2024-09-12T03:25:13.811275+0000","last_undegraded":"2024-09-12T03:25:13.811275+0000","last_fullsized":"2024-09-12T03:25:13.811275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:11.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857786+0000","last_change":"2024-09-12T03:24:29.335888+0000","last_active":"2024-09-12T03:25:20.857786+0000","last_peered":"2024-09-12T03:25:20.857786+0000","last_clean":"2024-09-12T03:25:20.857786+0000","last_became_active":"2024-09-12T03:24:29.335535+0000","last_became_peered":"2024-09-12T03:24:29.335535+0000","last_unstale":"2024-09-12T03:25:20.857786+0000","last_undegraded":"2024-09-12T03:25:20.857786+0000","last_fullsized":"2024-09-12T03:25:20.857786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:19.246874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811072+0000","last_change":"2024-09-12T03:24:40.696385+0000","last_active":"2024-09-12T03:25:13.811072+0000","last_peered":"2024-09-12T03:25:13.811072+0000","last_clean":"2024-09-12T03:25:13.811072+0000","last_became_active":"2024-09-12T03:24:40.695968+0000","last_became_peered":"2024-09-12T03:24:40.695968+0000","last_unstale":"2024-09-12T03:25:13.811072+0000","last_undegraded":"2024-09-12T03:25:13.811072+0000","last_fullsized":"2024-09-12T03:25:13.811072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:31.768769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886617+0000","last_change":"2024-09-12T03:24:31.655017+0000","last_active":"2024-09-12T03:25:13.886617+0000","last_peered":"2024-09-12T03:25:13.886617+0000","last_clean":"2024-09-12T03:25:13.886617+0000","last_became_active":"2024-09-12T03:24:31.654597+0000","last_became_peered":"2024-09-12T03:24:31.654597+0000","last_unstale":"2024-09-12T03:25:13.886617+0000","last_undegraded":"2024-09-12T03:25:13.886617+0000","last_fullsized":"2024-09-12T03:25:13.886617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:13.053238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:35.651055+0000","last_change":"2024-09-12T03:24:29.303991+0000","last_active":"2024-09-12T03:25:35.651055+0000","last_peered":"2024-09-12T03:25:35.651055+0000","last_clean":"2024-09-12T03:25:35.651055+0000","last_became_active":"2024-09-12T03:24:29.303682+0000","last_became_peered":"2024-09-12T03:24:29.303682+0000","last_unstale":"2024-09-12T03:25:35.651055+0000","last_undegraded":"2024-09-12T03:25:35.651055+0000","last_fullsized":"2024-09-12T03:25:35.651055+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:59.403461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886212+0000","last_change":"2024-09-12T03:24:40.704304+0000","last_active":"2024-09-12T03:25:13.886212+0000","last_peered":"2024-09-12T03:25:13.886212+0000","last_clean":"2024-09-12T03:25:13.886212+0000","last_became_active":"2024-09-12T03:24:40.703967+0000","last_became_peered":"2024-09-12T03:24:40.703967+0000","last_unstale":"2024-09-12T03:25:13.886212+0000","last_undegraded":"2024-09-12T03:25:13.886212+0000","last_fullsized":"2024-09-12T03:25:13.886212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:00.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942398+0000","last_change":"2024-09-12T03:24:31.650303+0000","last_active":"2024-09-12T03:25:13.942398+0000","last_peered":"2024-09-12T03:25:13.942398+0000","last_clean":"2024-09-12T03:25:13.942398+0000","last_became_active":"2024-09-12T03:24:31.649223+0000","last_became_peered":"2024-09-12T03:24:31.649223+0000","last_unstale":"2024-09-12T03:25:13.942398+0000","last_undegraded":"2024-09-12T03:25:13.942398+0000","last_fullsized":"2024-09-12T03:25:13.942398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:45.190363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857037+0000","last_change":"2024-09-12T03:24:29.310570+0000","last_active":"2024-09-12T03:25:20.857037+0000","last_peered":"2024-09-12T03:25:20.857037+0000","last_clean":"2024-09-12T03:25:20.857037+0000","last_became_active":"2024-09-12T03:24:29.310249+0000","last_became_peered":"2024-09-12T03:24:29.310249+0000","last_unstale":"2024-09-12T03:25:20.857037+0000","last_undegraded":"2024-09-12T03:25:20.857037+0000","last_fullsized":"2024-09-12T03:25:20.857037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.049566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265053+0000","last_change":"2024-09-12T03:24:40.696716+0000","last_active":"2024-09-12T03:25:14.265053+0000","last_peered":"2024-09-12T03:25:14.265053+0000","last_clean":"2024-09-12T03:25:14.265053+0000","last_became_active":"2024-09-12T03:24:40.696322+0000","last_became_peered":"2024-09-12T03:24:40.696322+0000","last_unstale":"2024-09-12T03:25:14.265053+0000","last_undegraded":"2024-09-12T03:25:14.265053+0000","last_fullsized":"2024-09-12T03:25:14.265053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:31.315393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809413+0000","last_change":"2024-09-12T03:24:31.652360+0000","last_active":"2024-09-12T03:25:13.809413+0000","last_peered":"2024-09-12T03:25:13.809413+0000","last_clean":"2024-09-12T03:25:13.809413+0000","last_became_active":"2024-09-12T03:24:31.652066+0000","last_became_peered":"2024-09-12T03:24:31.652066+0000","last_unstale":"2024-09-12T03:25:13.809413+0000","last_undegraded":"2024-09-12T03:25:13.809413+0000","last_fullsized":"2024-09-12T03:25:13.809413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:54:15.312876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808499+0000","last_change":"2024-09-12T03:24:29.305811+0000","last_active":"2024-09-12T03:25:13.808499+0000","last_peered":"2024-09-12T03:25:13.808499+0000","last_clean":"2024-09-12T03:25:13.808499+0000","last_became_active":"2024-09-12T03:24:29.305531+0000","last_became_peered":"2024-09-12T03:24:29.305531+0000","last_unstale":"2024-09-12T03:25:13.808499+0000","last_undegraded":"2024-09-12T03:25:13.808499+0000","last_fullsized":"2024-09-12T03:25:13.808499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:08.496670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.267079+0000","last_change":"2024-09-12T03:25:14.267079+0000","last_active":"2024-09-12T03:25:14.267079+0000","last_peered":"2024-09-12T03:25:14.267079+0000","last_clean":"2024-09-12T03:25:14.267079+0000","last_became_active":"2024-09-12T03:25:14.266625+0000","last_became_peered":"2024-09-12T03:25:14.266625+0000","last_unstale":"2024-09-12T03:25:14.267079+0000","last_undegraded":"2024-09-12T03:25:14.267079+0000","last_fullsized":"2024-09-12T03:25:14.267079+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812997+0000","last_change":"2024-09-12T03:25:13.812997+0000","last_active":"2024-09-12T03:25:13.812997+0000","last_peered":"2024-09-12T03:25:13.812997+0000","last_clean":"2024-09-12T03:25:13.812997+0000","last_became_active":"2024-09-12T03:25:13.812596+0000","last_became_peered":"2024-09-12T03:25:13.812596+0000","last_unstale":"2024-09-12T03:25:13.812997+0000","last_undegraded":"2024-09-12T03:25:13.812997+0000","last_fullsized":"2024-09-12T03:25:13.812997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:43.228416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789061+0000","last_change":"2024-09-12T03:24:29.301269+0000","last_active":"2024-09-12T03:25:14.789061+0000","last_peered":"2024-09-12T03:25:14.789061+0000","last_clean":"2024-09-12T03:25:14.789061+0000","last_became_active":"2024-09-12T03:24:29.300937+0000","last_became_peered":"2024-09-12T03:24:29.300937+0000","last_unstale":"2024-09-12T03:25:14.789061+0000","last_undegraded":"2024-09-12T03:25:14.789061+0000","last_fullsized":"2024-09-12T03:25:14.789061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:24.453709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942183+0000","last_change":"2024-09-12T03:24:40.708631+0000","last_active":"2024-09-12T03:25:13.942183+0000","last_peered":"2024-09-12T03:25:13.942183+0000","last_clean":"2024-09-12T03:25:13.942183+0000","last_became_active":"2024-09-12T03:24:40.706381+0000","last_became_peered":"2024-09-12T03:24:40.706381+0000","last_unstale":"2024-09-12T03:25:13.942183+0000","last_undegraded":"2024-09-12T03:25:13.942183+0000","last_fullsized":"2024-09-12T03:25:13.942183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:10.945538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319147+0000","last_change":"2024-09-12T03:24:31.652471+0000","last_active":"2024-09-12T03:25:14.319147+0000","last_peered":"2024-09-12T03:25:14.319147+0000","last_clean":"2024-09-12T03:25:14.319147+0000","last_became_active":"2024-09-12T03:24:31.651566+0000","last_became_peered":"2024-09-12T03:24:31.651566+0000","last_unstale":"2024-09-12T03:25:14.319147+0000","last_undegraded":"2024-09-12T03:25:14.319147+0000","last_fullsized":"2024-09-12T03:25:14.319147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:01.238554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:36.887572+0000","last_change":"2024-09-12T03:24:29.306214+0000","last_active":"2024-09-12T03:25:36.887572+0000","last_peered":"2024-09-12T03:25:36.887572+0000","last_clean":"2024-09-12T03:25:36.887572+0000","last_became_active":"2024-09-12T03:24:29.305846+0000","last_became_peered":"2024-09-12T03:24:29.305846+0000","last_unstale":"2024-09-12T03:25:36.887572+0000","last_undegraded":"2024-09-12T03:25:36.887572+0000","last_fullsized":"2024-09-12T03:25:36.887572+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:50.940377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942373+0000","last_change":"2024-09-12T03:24:40.709412+0000","last_active":"2024-09-12T03:25:13.942373+0000","last_peered":"2024-09-12T03:25:13.942373+0000","last_clean":"2024-09-12T03:25:13.942373+0000","last_became_active":"2024-09-12T03:24:40.705106+0000","last_became_peered":"2024-09-12T03:24:40.705106+0000","last_unstale":"2024-09-12T03:25:13.942373+0000","last_undegraded":"2024-09-12T03:25:13.942373+0000","last_fullsized":"2024-09-12T03:25:13.942373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:32.963060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941696+0000","last_change":"2024-09-12T03:24:31.644654+0000","last_active":"2024-09-12T03:25:13.941696+0000","last_peered":"2024-09-12T03:25:13.941696+0000","last_clean":"2024-09-12T03:25:13.941696+0000","last_became_active":"2024-09-12T03:24:31.642477+0000","last_became_peered":"2024-09-12T03:24:31.642477+0000","last_unstale":"2024-09-12T03:25:13.941696+0000","last_undegraded":"2024-09-12T03:25:13.941696+0000","last_fullsized":"2024-09-12T03:25:13.941696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.758784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.862588+0000","last_change":"2024-09-12T03:24:29.306850+0000","last_active":"2024-09-12T03:25:21.862588+0000","last_peered":"2024-09-12T03:25:21.862588+0000","last_clean":"2024-09-12T03:25:21.862588+0000","last_became_active":"2024-09-12T03:24:29.305902+0000","last_became_peered":"2024-09-12T03:24:29.305902+0000","last_unstale":"2024-09-12T03:25:21.862588+0000","last_undegraded":"2024-09-12T03:25:21.862588+0000","last_fullsized":"2024-09-12T03:25:21.862588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:02.564521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320379+0000","last_change":"2024-09-12T03:24:40.700966+0000","last_active":"2024-09-12T03:25:14.320379+0000","last_peered":"2024-09-12T03:25:14.320379+0000","last_clean":"2024-09-12T03:25:14.320379+0000","last_became_active":"2024-09-12T03:24:40.700553+0000","last_became_peered":"2024-09-12T03:24:40.700553+0000","last_unstale":"2024-09-12T03:25:14.320379+0000","last_undegraded":"2024-09-12T03:25:14.320379+0000","last_fullsized":"2024-09-12T03:25:14.320379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:14.855459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886794+0000","last_change":"2024-09-12T03:24:31.653431+0000","last_active":"2024-09-12T03:25:13.886794+0000","last_peered":"2024-09-12T03:25:13.886794+0000","last_clean":"2024-09-12T03:25:13.886794+0000","last_became_active":"2024-09-12T03:24:31.648771+0000","last_became_peered":"2024-09-12T03:24:31.648771+0000","last_unstale":"2024-09-12T03:25:13.886794+0000","last_undegraded":"2024-09-12T03:25:13.886794+0000","last_fullsized":"2024-09-12T03:25:13.886794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:54.052828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:22.866402+0000","last_change":"2024-09-12T03:24:29.305825+0000","last_active":"2024-09-12T03:25:22.866402+0000","last_peered":"2024-09-12T03:25:22.866402+0000","last_clean":"2024-09-12T03:25:22.866402+0000","last_became_active":"2024-09-12T03:24:29.305468+0000","last_became_peered":"2024-09-12T03:24:29.305468+0000","last_unstale":"2024-09-12T03:25:22.866402+0000","last_undegraded":"2024-09-12T03:25:22.866402+0000","last_fullsized":"2024-09-12T03:25:22.866402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:30.899307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":363,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788894+0000","last_change":"2024-09-12T03:23:13.507232+0000","last_active":"2024-09-12T03:25:14.788894+0000","last_peered":"2024-09-12T03:25:14.788894+0000","last_clean":"2024-09-12T03:25:14.788894+0000","last_became_active":"2024-09-12T03:23:13.506683+0000","last_became_peered":"2024-09-12T03:23:13.506683+0000","last_unstale":"2024-09-12T03:25:14.788894+0000","last_undegraded":"2024-09-12T03:25:14.788894+0000","last_fullsized":"2024-09-12T03:25:14.788894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_clean_scrub_stamp":"2024-09-12T03:19:33.594997+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.684658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789197+0000","last_change":"2024-09-12T03:24:40.696812+0000","last_active":"2024-09-12T03:25:14.789197+0000","last_peered":"2024-09-12T03:25:14.789197+0000","last_clean":"2024-09-12T03:25:14.789197+0000","last_became_active":"2024-09-12T03:24:40.695586+0000","last_became_peered":"2024-09-12T03:24:40.695586+0000","last_unstale":"2024-09-12T03:25:14.789197+0000","last_undegraded":"2024-09-12T03:25:14.789197+0000","last_fullsized":"2024-09-12T03:25:14.789197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:10.086113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789290+0000","last_change":"2024-09-12T03:24:31.644517+0000","last_active":"2024-09-12T03:25:14.789290+0000","last_peered":"2024-09-12T03:25:14.789290+0000","last_clean":"2024-09-12T03:25:14.789290+0000","last_became_active":"2024-09-12T03:24:31.643996+0000","last_became_peered":"2024-09-12T03:24:31.643996+0000","last_unstale":"2024-09-12T03:25:14.789290+0000","last_undegraded":"2024-09-12T03:25:14.789290+0000","last_fullsized":"2024-09-12T03:25:14.789290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:57.543397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.862918+0000","last_change":"2024-09-12T03:25:14.266550+0000","last_active":"2024-09-12T03:25:21.862918+0000","last_peered":"2024-09-12T03:25:21.862918+0000","last_clean":"2024-09-12T03:25:21.862918+0000","last_became_active":"2024-09-12T03:25:14.266202+0000","last_became_peered":"2024-09-12T03:25:14.266202+0000","last_unstale":"2024-09-12T03:25:21.862918+0000","last_undegraded":"2024-09-12T03:25:21.862918+0000","last_fullsized":"2024-09-12T03:25:21.862918+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:00.981147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815001+0000","last_change":"2024-09-12T03:24:40.697001+0000","last_active":"2024-09-12T03:25:13.815001+0000","last_peered":"2024-09-12T03:25:13.815001+0000","last_clean":"2024-09-12T03:25:13.815001+0000","last_became_active":"2024-09-12T03:24:40.696091+0000","last_became_peered":"2024-09-12T03:24:40.696091+0000","last_unstale":"2024-09-12T03:25:13.815001+0000","last_undegraded":"2024-09-12T03:25:13.815001+0000","last_fullsized":"2024-09-12T03:25:13.815001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:07.073034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789084+0000","last_change":"2024-09-12T03:24:31.655819+0000","last_active":"2024-09-12T03:25:14.789084+0000","last_peered":"2024-09-12T03:25:14.789084+0000","last_clean":"2024-09-12T03:25:14.789084+0000","last_became_active":"2024-09-12T03:24:31.655417+0000","last_became_peered":"2024-09-12T03:24:31.655417+0000","last_unstale":"2024-09-12T03:25:14.789084+0000","last_undegraded":"2024-09-12T03:25:14.789084+0000","last_fullsized":"2024-09-12T03:25:14.789084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:54.380047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320453+0000","last_change":"2024-09-12T03:24:29.303732+0000","last_active":"2024-09-12T03:25:14.320453+0000","last_peered":"2024-09-12T03:25:14.320453+0000","last_clean":"2024-09-12T03:25:14.320453+0000","last_became_active":"2024-09-12T03:24:29.302314+0000","last_became_peered":"2024-09-12T03:24:29.302314+0000","last_unstale":"2024-09-12T03:25:14.320453+0000","last_undegraded":"2024-09-12T03:25:14.320453+0000","last_fullsized":"2024-09-12T03:25:14.320453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:51.555510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.817846+0000","last_change":"2024-09-12T03:25:13.817846+0000","last_active":"2024-09-12T03:25:13.817846+0000","last_peered":"2024-09-12T03:25:13.817846+0000","last_clean":"2024-09-12T03:25:13.817846+0000","last_became_active":"2024-09-12T03:25:13.817102+0000","last_became_peered":"2024-09-12T03:25:13.817102+0000","last_unstale":"2024-09-12T03:25:13.817846+0000","last_undegraded":"2024-09-12T03:25:13.817846+0000","last_fullsized":"2024-09-12T03:25:13.817846+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:26.472731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789248+0000","last_change":"2024-09-12T03:24:31.645293+0000","last_active":"2024-09-12T03:25:14.789248+0000","last_peered":"2024-09-12T03:25:14.789248+0000","last_clean":"2024-09-12T03:25:14.789248+0000","last_became_active":"2024-09-12T03:24:31.644944+0000","last_became_peered":"2024-09-12T03:24:31.644944+0000","last_unstale":"2024-09-12T03:25:14.789248+0000","last_undegraded":"2024-09-12T03:25:14.789248+0000","last_fullsized":"2024-09-12T03:25:14.789248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:38.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942492+0000","last_change":"2024-09-12T03:24:29.302710+0000","last_active":"2024-09-12T03:25:13.942492+0000","last_peered":"2024-09-12T03:25:13.942492+0000","last_clean":"2024-09-12T03:25:13.942492+0000","last_became_active":"2024-09-12T03:24:29.302262+0000","last_became_peered":"2024-09-12T03:24:29.302262+0000","last_unstale":"2024-09-12T03:25:13.942492+0000","last_undegraded":"2024-09-12T03:25:13.942492+0000","last_fullsized":"2024-09-12T03:25:13.942492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:08.392390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.318966+0000","last_change":"2024-09-12T03:24:40.696164+0000","last_active":"2024-09-12T03:25:14.318966+0000","last_peered":"2024-09-12T03:25:14.318966+0000","last_clean":"2024-09-12T03:25:14.318966+0000","last_became_active":"2024-09-12T03:24:40.695786+0000","last_became_peered":"2024-09-12T03:24:40.695786+0000","last_unstale":"2024-09-12T03:25:14.318966+0000","last_undegraded":"2024-09-12T03:25:14.318966+0000","last_fullsized":"2024-09-12T03:25:14.318966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:27.063196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265366+0000","last_change":"2024-09-12T03:24:31.645341+0000","last_active":"2024-09-12T03:25:14.265366+0000","last_peered":"2024-09-12T03:25:14.265366+0000","last_clean":"2024-09-12T03:25:14.265366+0000","last_became_active":"2024-09-12T03:24:31.644496+0000","last_became_peered":"2024-09-12T03:24:31.644496+0000","last_unstale":"2024-09-12T03:25:14.265366+0000","last_undegraded":"2024-09-12T03:25:14.265366+0000","last_fullsized":"2024-09-12T03:25:14.265366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:09.416806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:22.866391+0000","last_change":"2024-09-12T03:24:29.305762+0000","last_active":"2024-09-12T03:25:22.866391+0000","last_peered":"2024-09-12T03:25:22.866391+0000","last_clean":"2024-09-12T03:25:22.866391+0000","last_became_active":"2024-09-12T03:24:29.305445+0000","last_became_peered":"2024-09-12T03:24:29.305445+0000","last_unstale":"2024-09-12T03:25:22.866391+0000","last_undegraded":"2024-09-12T03:25:22.866391+0000","last_fullsized":"2024-09-12T03:25:22.866391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:50.032960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319111+0000","last_change":"2024-09-12T03:24:40.703795+0000","last_active":"2024-09-12T03:25:14.319111+0000","last_peered":"2024-09-12T03:25:14.319111+0000","last_clean":"2024-09-12T03:25:14.319111+0000","last_became_active":"2024-09-12T03:24:40.703361+0000","last_became_peered":"2024-09-12T03:24:40.703361+0000","last_unstale":"2024-09-12T03:25:14.319111+0000","last_undegraded":"2024-09-12T03:25:14.319111+0000","last_fullsized":"2024-09-12T03:25:14.319111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:12.430431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319348+0000","last_change":"2024-09-12T03:24:31.649521+0000","last_active":"2024-09-12T03:25:14.319348+0000","last_peered":"2024-09-12T03:25:14.319348+0000","last_clean":"2024-09-12T03:25:14.319348+0000","last_became_active":"2024-09-12T03:24:31.649217+0000","last_became_peered":"2024-09-12T03:24:31.649217+0000","last_unstale":"2024-09-12T03:25:14.319348+0000","last_undegraded":"2024-09-12T03:25:14.319348+0000","last_fullsized":"2024-09-12T03:25:14.319348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:33.885153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.865005+0000","last_change":"2024-09-12T03:24:29.308568+0000","last_active":"2024-09-12T03:25:21.865005+0000","last_peered":"2024-09-12T03:25:21.865005+0000","last_clean":"2024-09-12T03:25:21.865005+0000","last_became_active":"2024-09-12T03:24:29.308263+0000","last_became_peered":"2024-09-12T03:24:29.308263+0000","last_unstale":"2024-09-12T03:25:21.865005+0000","last_undegraded":"2024-09-12T03:25:21.865005+0000","last_fullsized":"2024-09-12T03:25:21.865005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:54.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942390+0000","last_change":"2024-09-12T03:24:40.707719+0000","last_active":"2024-09-12T03:25:13.942390+0000","last_peered":"2024-09-12T03:25:13.942390+0000","last_clean":"2024-09-12T03:25:13.942390+0000","last_became_active":"2024-09-12T03:24:40.707364+0000","last_became_peered":"2024-09-12T03:24:40.707364+0000","last_unstale":"2024-09-12T03:25:13.942390+0000","last_undegraded":"2024-09-12T03:25:13.942390+0000","last_fullsized":"2024-09-12T03:25:13.942390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:00.423368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788656+0000","last_change":"2024-09-12T03:24:31.646006+0000","last_active":"2024-09-12T03:25:14.788656+0000","last_peered":"2024-09-12T03:25:14.788656+0000","last_clean":"2024-09-12T03:25:14.788656+0000","last_became_active":"2024-09-12T03:24:31.643165+0000","last_became_peered":"2024-09-12T03:24:31.643165+0000","last_unstale":"2024-09-12T03:25:14.788656+0000","last_undegraded":"2024-09-12T03:25:14.788656+0000","last_fullsized":"2024-09-12T03:25:14.788656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:33.883658+0000","last_change":"2024-09-12T03:24:29.304521+0000","last_active":"2024-09-12T03:25:33.883658+0000","last_peered":"2024-09-12T03:25:33.883658+0000","last_clean":"2024-09-12T03:25:33.883658+0000","last_became_active":"2024-09-12T03:24:29.304147+0000","last_became_peered":"2024-09-12T03:24:29.304147+0000","last_unstale":"2024-09-12T03:25:33.883658+0000","last_undegraded":"2024-09-12T03:25:33.883658+0000","last_fullsized":"2024-09-12T03:25:33.883658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:17.834282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808706+0000","last_change":"2024-09-12T03:24:40.706844+0000","last_active":"2024-09-12T03:25:13.808706+0000","last_peered":"2024-09-12T03:25:13.808706+0000","last_clean":"2024-09-12T03:25:13.808706+0000","last_became_active":"2024-09-12T03:24:40.704829+0000","last_became_peered":"2024-09-12T03:24:40.704829+0000","last_unstale":"2024-09-12T03:25:13.808706+0000","last_undegraded":"2024-09-12T03:25:13.808706+0000","last_fullsized":"2024-09-12T03:25:13.808706+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:32.875316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788924+0000","last_change":"2024-09-12T03:24:31.650168+0000","last_active":"2024-09-12T03:25:14.788924+0000","last_peered":"2024-09-12T03:25:14.788924+0000","last_clean":"2024-09-12T03:25:14.788924+0000","last_became_active":"2024-09-12T03:24:31.649754+0000","last_became_peered":"2024-09-12T03:24:31.649754+0000","last_unstale":"2024-09-12T03:25:14.788924+0000","last_undegraded":"2024-09-12T03:25:14.788924+0000","last_fullsized":"2024-09-12T03:25:14.788924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:09.615836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.858421+0000","last_change":"2024-09-12T03:24:29.303488+0000","last_active":"2024-09-12T03:25:20.858421+0000","last_peered":"2024-09-12T03:25:20.858421+0000","last_clean":"2024-09-12T03:25:20.858421+0000","last_became_active":"2024-09-12T03:24:29.301858+0000","last_became_peered":"2024-09-12T03:24:29.301858+0000","last_unstale":"2024-09-12T03:25:20.858421+0000","last_undegraded":"2024-09-12T03:25:20.858421+0000","last_fullsized":"2024-09-12T03:25:20.858421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:27.846820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265096+0000","last_change":"2024-09-12T03:24:40.697734+0000","last_active":"2024-09-12T03:25:14.265096+0000","last_peered":"2024-09-12T03:25:14.265096+0000","last_clean":"2024-09-12T03:25:14.265096+0000","last_became_active":"2024-09-12T03:24:40.697271+0000","last_became_peered":"2024-09-12T03:24:40.697271+0000","last_unstale":"2024-09-12T03:25:14.265096+0000","last_undegraded":"2024-09-12T03:25:14.265096+0000","last_fullsized":"2024-09-12T03:25:14.265096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.336322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941957+0000","last_change":"2024-09-12T03:24:31.647040+0000","last_active":"2024-09-12T03:25:13.941957+0000","last_peered":"2024-09-12T03:25:13.941957+0000","last_clean":"2024-09-12T03:25:13.941957+0000","last_became_active":"2024-09-12T03:24:31.643365+0000","last_became_peered":"2024-09-12T03:24:31.643365+0000","last_unstale":"2024-09-12T03:25:13.941957+0000","last_undegraded":"2024-09-12T03:25:13.941957+0000","last_fullsized":"2024-09-12T03:25:13.941957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:06.638005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.864927+0000","last_change":"2024-09-12T03:24:29.303390+0000","last_active":"2024-09-12T03:25:21.864927+0000","last_peered":"2024-09-12T03:25:21.864927+0000","last_clean":"2024-09-12T03:25:21.864927+0000","last_became_active":"2024-09-12T03:24:29.303072+0000","last_became_peered":"2024-09-12T03:24:29.303072+0000","last_unstale":"2024-09-12T03:25:21.864927+0000","last_undegraded":"2024-09-12T03:25:21.864927+0000","last_fullsized":"2024-09-12T03:25:21.864927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:01.572888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320143+0000","last_change":"2024-09-12T03:24:40.707977+0000","last_active":"2024-09-12T03:25:14.320143+0000","last_peered":"2024-09-12T03:25:14.320143+0000","last_clean":"2024-09-12T03:25:14.320143+0000","last_became_active":"2024-09-12T03:24:40.707497+0000","last_became_peered":"2024-09-12T03:24:40.707497+0000","last_unstale":"2024-09-12T03:25:14.320143+0000","last_undegraded":"2024-09-12T03:25:14.320143+0000","last_fullsized":"2024-09-12T03:25:14.320143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:45.474021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809346+0000","last_change":"2024-09-12T03:24:31.642417+0000","last_active":"2024-09-12T03:25:13.809346+0000","last_peered":"2024-09-12T03:25:13.809346+0000","last_clean":"2024-09-12T03:25:13.809346+0000","last_became_active":"2024-09-12T03:24:31.642076+0000","last_became_peered":"2024-09-12T03:24:31.642076+0000","last_unstale":"2024-09-12T03:25:13.809346+0000","last_undegraded":"2024-09-12T03:25:13.809346+0000","last_fullsized":"2024-09-12T03:25:13.809346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:56.436088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'25","reported_seq":89,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:33.886672+0000","last_change":"2024-09-12T03:24:29.302982+0000","last_active":"2024-09-12T03:25:33.886672+0000","last_peered":"2024-09-12T03:25:33.886672+0000","last_clean":"2024-09-12T03:25:33.886672+0000","last_became_active":"2024-09-12T03:24:29.302369+0000","last_became_peered":"2024-09-12T03:24:29.302369+0000","last_unstale":"2024-09-12T03:25:33.886672+0000","last_undegraded":"2024-09-12T03:25:33.886672+0000","last_fullsized":"2024-09-12T03:25:33.886672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:24:30.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":57,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808161+0000","last_change":"2024-09-12T03:24:40.696745+0000","last_active":"2024-09-12T03:25:13.808161+0000","last_peered":"2024-09-12T03:25:13.808161+0000","last_clean":"2024-09-12T03:25:13.808161+0000","last_became_active":"2024-09-12T03:24:40.696273+0000","last_became_peered":"2024-09-12T03:24:40.696273+0000","last_unstale":"2024-09-12T03:25:13.808161+0000","last_undegraded":"2024-09-12T03:25:13.808161+0000","last_fullsized":"2024-09-12T03:25:13.808161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.519765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815432+0000","last_change":"2024-09-12T03:24:31.650567+0000","last_active":"2024-09-12T03:25:13.815432+0000","last_peered":"2024-09-12T03:25:13.815432+0000","last_clean":"2024-09-12T03:25:13.815432+0000","last_became_active":"2024-09-12T03:24:31.650242+0000","last_became_peered":"2024-09-12T03:24:31.650242+0000","last_unstale":"2024-09-12T03:25:13.815432+0000","last_undegraded":"2024-09-12T03:25:13.815432+0000","last_fullsized":"2024-09-12T03:25:13.815432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:14.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.858384+0000","last_change":"2024-09-12T03:24:29.310630+0000","last_active":"2024-09-12T03:25:20.858384+0000","last_peered":"2024-09-12T03:25:20.858384+0000","last_clean":"2024-09-12T03:25:20.858384+0000","last_became_active":"2024-09-12T03:24:29.310302+0000","last_became_peered":"2024-09-12T03:24:29.310302+0000","last_unstale":"2024-09-12T03:25:20.858384+0000","last_undegraded":"2024-09-12T03:25:20.858384+0000","last_fullsized":"2024-09-12T03:25:20.858384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:53.835246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812536+0000","last_change":"2024-09-12T03:24:40.698941+0000","last_active":"2024-09-12T03:25:13.812536+0000","last_peered":"2024-09-12T03:25:13.812536+0000","last_clean":"2024-09-12T03:25:13.812536+0000","last_became_active":"2024-09-12T03:24:40.698323+0000","last_became_peered":"2024-09-12T03:24:40.698323+0000","last_unstale":"2024-09-12T03:25:13.812536+0000","last_undegraded":"2024-09-12T03:25:13.812536+0000","last_fullsized":"2024-09-12T03:25:13.812536+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:29.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320685+0000","last_change":"2024-09-12T03:25:14.320685+0000","last_active":"2024-09-12T03:25:14.320685+0000","last_peered":"2024-09-12T03:25:14.320685+0000","last_clean":"2024-09-12T03:25:14.320685+0000","last_became_active":"2024-09-12T03:25:14.320295+0000","last_became_peered":"2024-09-12T03:25:14.320295+0000","last_unstale":"2024-09-12T03:25:14.320685+0000","last_undegraded":"2024-09-12T03:25:14.320685+0000","last_fullsized":"2024-09-12T03:25:14.320685+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:42.459136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857188+0000","last_change":"2024-09-12T03:24:29.302988+0000","last_active":"2024-09-12T03:25:20.857188+0000","last_peered":"2024-09-12T03:25:20.857188+0000","last_clean":"2024-09-12T03:25:20.857188+0000","last_became_active":"2024-09-12T03:24:29.302569+0000","last_became_peered":"2024-09-12T03:24:29.302569+0000","last_unstale":"2024-09-12T03:25:20.857188+0000","last_undegraded":"2024-09-12T03:25:20.857188+0000","last_fullsized":"2024-09-12T03:25:20.857188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:00.196284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.807973+0000","last_change":"2024-09-12T03:24:40.701486+0000","last_active":"2024-09-12T03:25:13.807973+0000","last_peered":"2024-09-12T03:25:13.807973+0000","last_clean":"2024-09-12T03:25:13.807973+0000","last_became_active":"2024-09-12T03:24:40.701078+0000","last_became_peered":"2024-09-12T03:24:40.701078+0000","last_unstale":"2024-09-12T03:25:13.807973+0000","last_undegraded":"2024-09-12T03:25:13.807973+0000","last_fullsized":"2024-09-12T03:25:13.807973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:50.032773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815679+0000","last_change":"2024-09-12T03:24:31.646681+0000","last_active":"2024-09-12T03:25:13.815679+0000","last_peered":"2024-09-12T03:25:13.815679+0000","last_clean":"2024-09-12T03:25:13.815679+0000","last_became_active":"2024-09-12T03:24:31.646389+0000","last_became_peered":"2024-09-12T03:24:31.646389+0000","last_unstale":"2024-09-12T03:25:13.815679+0000","last_undegraded":"2024-09-12T03:25:13.815679+0000","last_fullsized":"2024-09-12T03:25:13.815679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:37.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:32.882917+0000","last_change":"2024-09-12T03:24:29.299316+0000","last_active":"2024-09-12T03:25:32.882917+0000","last_peered":"2024-09-12T03:25:32.882917+0000","last_clean":"2024-09-12T03:25:32.882917+0000","last_became_active":"2024-09-12T03:24:29.298900+0000","last_became_peered":"2024-09-12T03:24:29.298900+0000","last_unstale":"2024-09-12T03:25:32.882917+0000","last_undegraded":"2024-09-12T03:25:32.882917+0000","last_fullsized":"2024-09-12T03:25:32.882917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:36.650938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886404+0000","last_change":"2024-09-12T03:24:40.702034+0000","last_active":"2024-09-12T03:25:13.886404+0000","last_peered":"2024-09-12T03:25:13.886404+0000","last_clean":"2024-09-12T03:25:13.886404+0000","last_became_active":"2024-09-12T03:24:40.699312+0000","last_became_peered":"2024-09-12T03:24:40.699312+0000","last_unstale":"2024-09-12T03:25:13.886404+0000","last_undegraded":"2024-09-12T03:25:13.886404+0000","last_fullsized":"2024-09-12T03:25:13.886404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:58.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811172+0000","last_change":"2024-09-12T03:24:40.701744+0000","last_active":"2024-09-12T03:25:13.811172+0000","last_peered":"2024-09-12T03:25:13.811172+0000","last_clean":"2024-09-12T03:25:13.811172+0000","last_became_active":"2024-09-12T03:24:40.700929+0000","last_became_peered":"2024-09-12T03:24:40.700929+0000","last_unstale":"2024-09-12T03:25:13.811172+0000","last_undegraded":"2024-09-12T03:25:13.811172+0000","last_fullsized":"2024-09-12T03:25:13.811172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.851810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808040+0000","last_change":"2024-09-12T03:24:31.647774+0000","last_active":"2024-09-12T03:25:13.808040+0000","last_peered":"2024-09-12T03:25:13.808040+0000","last_clean":"2024-09-12T03:25:13.808040+0000","last_became_active":"2024-09-12T03:24:31.647329+0000","last_became_peered":"2024-09-12T03:24:31.647329+0000","last_unstale":"2024-09-12T03:25:13.808040+0000","last_undegraded":"2024-09-12T03:25:13.808040+0000","last_fullsized":"2024-09-12T03:25:13.808040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:00.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319360+0000","last_change":"2024-09-12T03:24:29.307455+0000","last_active":"2024-09-12T03:25:14.319360+0000","last_peered":"2024-09-12T03:25:14.319360+0000","last_clean":"2024-09-12T03:25:14.319360+0000","last_became_active":"2024-09-12T03:24:29.307120+0000","last_became_peered":"2024-09-12T03:24:29.307120+0000","last_unstale":"2024-09-12T03:25:14.319360+0000","last_undegraded":"2024-09-12T03:25:14.319360+0000","last_fullsized":"2024-09-12T03:25:14.319360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:29.374097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265451+0000","last_change":"2024-09-12T03:24:40.695149+0000","last_active":"2024-09-12T03:25:14.265451+0000","last_peered":"2024-09-12T03:25:14.265451+0000","last_clean":"2024-09-12T03:25:14.265451+0000","last_became_active":"2024-09-12T03:24:40.694755+0000","last_became_peered":"2024-09-12T03:24:40.694755+0000","last_unstale":"2024-09-12T03:25:14.265451+0000","last_undegraded":"2024-09-12T03:25:14.265451+0000","last_fullsized":"2024-09-12T03:25:14.265451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:34.045481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886493+0000","last_change":"2024-09-12T03:24:31.654674+0000","last_active":"2024-09-12T03:25:13.886493+0000","last_peered":"2024-09-12T03:25:13.886493+0000","last_clean":"2024-09-12T03:25:13.886493+0000","last_became_active":"2024-09-12T03:24:31.654319+0000","last_became_peered":"2024-09-12T03:24:31.654319+0000","last_unstale":"2024-09-12T03:25:13.886493+0000","last_undegraded":"2024-09-12T03:25:13.886493+0000","last_fullsized":"2024-09-12T03:25:13.886493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:33.441162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.864549+0000","last_change":"2024-09-12T03:24:29.307849+0000","last_active":"2024-09-12T03:25:21.864549+0000","last_peered":"2024-09-12T03:25:21.864549+0000","last_clean":"2024-09-12T03:25:21.864549+0000","last_became_active":"2024-09-12T03:24:29.306017+0000","last_became_peered":"2024-09-12T03:24:29.306017+0000","last_unstale":"2024-09-12T03:25:21.864549+0000","last_undegraded":"2024-09-12T03:25:21.864549+0000","last_fullsized":"2024-09-12T03:25:21.864549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:37.478289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.865430+0000","last_change":"2024-09-12T03:24:29.302410+0000","last_active":"2024-09-12T03:25:21.865430+0000","last_peered":"2024-09-12T03:25:21.865430+0000","last_clean":"2024-09-12T03:25:21.865430+0000","last_became_active":"2024-09-12T03:24:29.302031+0000","last_became_peered":"2024-09-12T03:24:29.302031+0000","last_unstale":"2024-09-12T03:25:21.865430+0000","last_undegraded":"2024-09-12T03:25:21.865430+0000","last_fullsized":"2024-09-12T03:25:21.865430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:11.698690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319976+0000","last_change":"2024-09-12T03:24:31.652823+0000","last_active":"2024-09-12T03:25:14.319976+0000","last_peered":"2024-09-12T03:25:14.319976+0000","last_clean":"2024-09-12T03:25:14.319976+0000","last_became_active":"2024-09-12T03:24:31.652438+0000","last_became_peered":"2024-09-12T03:24:31.652438+0000","last_unstale":"2024-09-12T03:25:14.319976+0000","last_undegraded":"2024-09-12T03:25:14.319976+0000","last_fullsized":"2024-09-12T03:25:14.319976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942339+0000","last_change":"2024-09-12T03:25:13.942339+0000","last_active":"2024-09-12T03:25:13.942339+0000","last_peered":"2024-09-12T03:25:13.942339+0000","last_clean":"2024-09-12T03:25:13.942339+0000","last_became_active":"2024-09-12T03:25:13.942030+0000","last_became_peered":"2024-09-12T03:25:13.942030+0000","last_unstale":"2024-09-12T03:25:13.942339+0000","last_undegraded":"2024-09-12T03:25:13.942339+0000","last_fullsized":"2024-09-12T03:25:13.942339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:39.584996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:32.883600+0000","last_change":"2024-09-12T03:24:29.303318+0000","last_active":"2024-09-12T03:25:32.883600+0000","last_peered":"2024-09-12T03:25:32.883600+0000","last_clean":"2024-09-12T03:25:32.883600+0000","last_became_active":"2024-09-12T03:24:29.303001+0000","last_became_peered":"2024-09-12T03:24:29.303001+0000","last_unstale":"2024-09-12T03:25:32.883600+0000","last_undegraded":"2024-09-12T03:25:32.883600+0000","last_fullsized":"2024-09-12T03:25:32.883600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:08.094697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810264+0000","last_change":"2024-09-12T03:24:31.644465+0000","last_active":"2024-09-12T03:25:13.810264+0000","last_peered":"2024-09-12T03:25:13.810264+0000","last_clean":"2024-09-12T03:25:13.810264+0000","last_became_active":"2024-09-12T03:24:31.644141+0000","last_became_peered":"2024-09-12T03:24:31.644141+0000","last_unstale":"2024-09-12T03:25:13.810264+0000","last_undegraded":"2024-09-12T03:25:13.810264+0000","last_fullsized":"2024-09-12T03:25:13.810264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:37.448614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811203+0000","last_change":"2024-09-12T03:24:40.695190+0000","last_active":"2024-09-12T03:25:13.811203+0000","last_peered":"2024-09-12T03:25:13.811203+0000","last_clean":"2024-09-12T03:25:13.811203+0000","last_became_active":"2024-09-12T03:24:40.694746+0000","last_became_peered":"2024-09-12T03:24:40.694746+0000","last_unstale":"2024-09-12T03:25:13.811203+0000","last_undegraded":"2024-09-12T03:25:13.811203+0000","last_fullsized":"2024-09-12T03:25:13.811203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:31.905437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265069+0000","last_change":"2024-09-12T03:24:31.649068+0000","last_active":"2024-09-12T03:25:14.265069+0000","last_peered":"2024-09-12T03:25:14.265069+0000","last_clean":"2024-09-12T03:25:14.265069+0000","last_became_active":"2024-09-12T03:24:31.648761+0000","last_became_peered":"2024-09-12T03:24:31.648761+0000","last_unstale":"2024-09-12T03:25:14.265069+0000","last_undegraded":"2024-09-12T03:25:14.265069+0000","last_fullsized":"2024-09-12T03:25:14.265069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:08.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811318+0000","last_change":"2024-09-12T03:24:31.643604+0000","last_active":"2024-09-12T03:25:13.811318+0000","last_peered":"2024-09-12T03:25:13.811318+0000","last_clean":"2024-09-12T03:25:13.811318+0000","last_became_active":"2024-09-12T03:24:31.643053+0000","last_became_peered":"2024-09-12T03:24:31.643053+0000","last_unstale":"2024-09-12T03:25:13.811318+0000","last_undegraded":"2024-09-12T03:25:13.811318+0000","last_fullsized":"2024-09-12T03:25:13.811318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:59.814891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810842+0000","last_change":"2024-09-12T03:24:31.648376+0000","last_active":"2024-09-12T03:25:13.810842+0000","last_peered":"2024-09-12T03:25:13.810842+0000","last_clean":"2024-09-12T03:25:13.810842+0000","last_became_active":"2024-09-12T03:24:31.648076+0000","last_became_peered":"2024-09-12T03:24:31.648076+0000","last_unstale":"2024-09-12T03:25:13.810842+0000","last_undegraded":"2024-09-12T03:25:13.810842+0000","last_fullsized":"2024-09-12T03:25:13.810842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:47.736362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810502+0000","last_change":"2024-09-12T03:24:31.652489+0000","last_active":"2024-09-12T03:25:13.810502+0000","last_peered":"2024-09-12T03:25:13.810502+0000","last_clean":"2024-09-12T03:25:13.810502+0000","last_became_active":"2024-09-12T03:24:31.651907+0000","last_became_peered":"2024-09-12T03:24:31.651907+0000","last_unstale":"2024-09-12T03:25:13.810502+0000","last_undegraded":"2024-09-12T03:25:13.810502+0000","last_fullsized":"2024-09-12T03:25:13.810502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:24.584264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942284+0000","last_change":"2024-09-12T03:24:31.649967+0000","last_active":"2024-09-12T03:25:13.942284+0000","last_peered":"2024-09-12T03:25:13.942284+0000","last_clean":"2024-09-12T03:25:13.942284+0000","last_became_active":"2024-09-12T03:24:31.649687+0000","last_became_peered":"2024-09-12T03:24:31.649687+0000","last_unstale":"2024-09-12T03:25:13.942284+0000","last_undegraded":"2024-09-12T03:25:13.942284+0000","last_fullsized":"2024-09-12T03:25:13.942284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:14.875352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265499+0000","last_change":"2024-09-12T03:24:31.645115+0000","last_active":"2024-09-12T03:25:14.265499+0000","last_peered":"2024-09-12T03:25:14.265499+0000","last_clean":"2024-09-12T03:25:14.265499+0000","last_became_active":"2024-09-12T03:24:31.644442+0000","last_became_peered":"2024-09-12T03:24:31.644442+0000","last_unstale":"2024-09-12T03:25:14.265499+0000","last_undegraded":"2024-09-12T03:25:14.265499+0000","last_fullsized":"2024-09-12T03:25:14.265499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:28.539948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812088+0000","last_change":"2024-09-12T03:24:31.643283+0000","last_active":"2024-09-12T03:25:13.812088+0000","last_peered":"2024-09-12T03:25:13.812088+0000","last_clean":"2024-09-12T03:25:13.812088+0000","last_became_active":"2024-09-12T03:24:31.642935+0000","last_became_peered":"2024-09-12T03:24:31.642935+0000","last_unstale":"2024-09-12T03:25:13.812088+0000","last_undegraded":"2024-09-12T03:25:13.812088+0000","last_fullsized":"2024-09-12T03:25:13.812088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:30.754875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.817610+0000","last_change":"2024-09-12T03:25:13.817610+0000","last_active":"2024-09-12T03:25:13.817610+0000","last_peered":"2024-09-12T03:25:13.817610+0000","last_clean":"2024-09-12T03:25:13.817610+0000","last_became_active":"2024-09-12T03:25:13.817008+0000","last_became_peered":"2024-09-12T03:25:13.817008+0000","last_unstale":"2024-09-12T03:25:13.817610+0000","last_undegraded":"2024-09-12T03:25:13.817610+0000","last_fullsized":"2024-09-12T03:25:13.817610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:45.334353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319122+0000","last_change":"2024-09-12T03:24:31.656588+0000","last_active":"2024-09-12T03:25:14.319122+0000","last_peered":"2024-09-12T03:25:14.319122+0000","last_clean":"2024-09-12T03:25:14.319122+0000","last_became_active":"2024-09-12T03:24:31.656294+0000","last_became_peered":"2024-09-12T03:24:31.656294+0000","last_unstale":"2024-09-12T03:25:14.319122+0000","last_undegraded":"2024-09-12T03:25:14.319122+0000","last_fullsized":"2024-09-12T03:25:14.319122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.855802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886506+0000","last_change":"2024-09-12T03:24:31.648986+0000","last_active":"2024-09-12T03:25:13.886506+0000","last_peered":"2024-09-12T03:25:13.886506+0000","last_clean":"2024-09-12T03:25:13.886506+0000","last_became_active":"2024-09-12T03:24:31.648144+0000","last_became_peered":"2024-09-12T03:24:31.648144+0000","last_unstale":"2024-09-12T03:25:13.886506+0000","last_undegraded":"2024-09-12T03:25:13.886506+0000","last_fullsized":"2024-09-12T03:25:13.886506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.823742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815183+0000","last_change":"2024-09-12T03:24:31.644390+0000","last_active":"2024-09-12T03:25:13.815183+0000","last_peered":"2024-09-12T03:25:13.815183+0000","last_clean":"2024-09-12T03:25:13.815183+0000","last_became_active":"2024-09-12T03:24:31.642660+0000","last_became_peered":"2024-09-12T03:24:31.642660+0000","last_unstale":"2024-09-12T03:25:13.815183+0000","last_undegraded":"2024-09-12T03:25:13.815183+0000","last_fullsized":"2024-09-12T03:25:13.815183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:58.577883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941756+0000","last_change":"2024-09-12T03:24:31.649895+0000","last_active":"2024-09-12T03:25:13.941756+0000","last_peered":"2024-09-12T03:25:13.941756+0000","last_clean":"2024-09-12T03:25:13.941756+0000","last_became_active":"2024-09-12T03:24:31.649141+0000","last_became_peered":"2024-09-12T03:24:31.649141+0000","last_unstale":"2024-09-12T03:25:13.941756+0000","last_undegraded":"2024-09-12T03:25:13.941756+0000","last_fullsized":"2024-09-12T03:25:13.941756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:48.175496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265330+0000","last_change":"2024-09-12T03:24:31.640402+0000","last_active":"2024-09-12T03:25:14.265330+0000","last_peered":"2024-09-12T03:25:14.265330+0000","last_clean":"2024-09-12T03:25:14.265330+0000","last_became_active":"2024-09-12T03:24:31.639987+0000","last_became_peered":"2024-09-12T03:24:31.639987+0000","last_unstale":"2024-09-12T03:25:14.265330+0000","last_undegraded":"2024-09-12T03:25:14.265330+0000","last_fullsized":"2024-09-12T03:25:14.265330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:51.540149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811506+0000","last_change":"2024-09-12T03:24:31.647524+0000","last_active":"2024-09-12T03:25:13.811506+0000","last_peered":"2024-09-12T03:25:13.811506+0000","last_clean":"2024-09-12T03:25:13.811506+0000","last_became_active":"2024-09-12T03:24:31.647204+0000","last_became_peered":"2024-09-12T03:24:31.647204+0000","last_unstale":"2024-09-12T03:25:13.811506+0000","last_undegraded":"2024-09-12T03:25:13.811506+0000","last_fullsized":"2024-09-12T03:25:13.811506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.895586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808709+0000","last_change":"2024-09-12T03:24:31.640797+0000","last_active":"2024-09-12T03:25:13.808709+0000","last_peered":"2024-09-12T03:25:13.808709+0000","last_clean":"2024-09-12T03:25:13.808709+0000","last_became_active":"2024-09-12T03:24:31.640479+0000","last_became_peered":"2024-09-12T03:24:31.640479+0000","last_unstale":"2024-09-12T03:25:13.808709+0000","last_undegraded":"2024-09-12T03:25:13.808709+0000","last_fullsized":"2024-09-12T03:25:13.808709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:08.762939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265755+0000","last_change":"2024-09-12T03:24:31.644225+0000","last_active":"2024-09-12T03:25:14.265755+0000","last_peered":"2024-09-12T03:25:14.265755+0000","last_clean":"2024-09-12T03:25:14.265755+0000","last_became_active":"2024-09-12T03:24:31.643877+0000","last_became_peered":"2024-09-12T03:24:31.643877+0000","last_unstale":"2024-09-12T03:25:14.265755+0000","last_undegraded":"2024-09-12T03:25:14.265755+0000","last_fullsized":"2024-09-12T03:25:14.265755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:50.511392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.814692+0000","last_change":"2024-09-12T03:24:31.644917+0000","last_active":"2024-09-12T03:25:13.814692+0000","last_peered":"2024-09-12T03:25:13.814692+0000","last_clean":"2024-09-12T03:25:13.814692+0000","last_became_active":"2024-09-12T03:24:31.644591+0000","last_became_peered":"2024-09-12T03:24:31.644591+0000","last_unstale":"2024-09-12T03:25:13.814692+0000","last_undegraded":"2024-09-12T03:25:13.814692+0000","last_fullsized":"2024-09-12T03:25:13.814692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:21.961959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810723+0000","last_change":"2024-09-12T03:24:31.648146+0000","last_active":"2024-09-12T03:25:13.810723+0000","last_peered":"2024-09-12T03:25:13.810723+0000","last_clean":"2024-09-12T03:25:13.810723+0000","last_became_active":"2024-09-12T03:24:31.641423+0000","last_became_peered":"2024-09-12T03:24:31.641423+0000","last_unstale":"2024-09-12T03:25:13.810723+0000","last_undegraded":"2024-09-12T03:25:13.810723+0000","last_fullsized":"2024-09-12T03:25:13.810723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:50.985126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811003+0000","last_change":"2024-09-12T03:24:31.641030+0000","last_active":"2024-09-12T03:25:13.811003+0000","last_peered":"2024-09-12T03:25:13.811003+0000","last_clean":"2024-09-12T03:25:13.811003+0000","last_became_active":"2024-09-12T03:24:31.639015+0000","last_became_peered":"2024-09-12T03:24:31.639015+0000","last_unstale":"2024-09-12T03:25:13.811003+0000","last_undegraded":"2024-09-12T03:25:13.811003+0000","last_fullsized":"2024-09-12T03:25:13.811003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.961304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941758+0000","last_change":"2024-09-12T03:24:31.644601+0000","last_active":"2024-09-12T03:25:13.941758+0000","last_peered":"2024-09-12T03:25:13.941758+0000","last_clean":"2024-09-12T03:25:13.941758+0000","last_became_active":"2024-09-12T03:24:31.644122+0000","last_became_peered":"2024-09-12T03:24:31.644122+0000","last_unstale":"2024-09-12T03:25:13.941758+0000","last_undegraded":"2024-09-12T03:25:13.941758+0000","last_fullsized":"2024-09-12T03:25:13.941758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:06.335464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813711+0000","last_change":"2024-09-12T03:24:31.641496+0000","last_active":"2024-09-12T03:25:13.813711+0000","last_peered":"2024-09-12T03:25:13.813711+0000","last_clean":"2024-09-12T03:25:13.813711+0000","last_became_active":"2024-09-12T03:24:31.641195+0000","last_became_peered":"2024-09-12T03:24:31.641195+0000","last_unstale":"2024-09-12T03:25:13.813711+0000","last_undegraded":"2024-09-12T03:25:13.813711+0000","last_fullsized":"2024-09-12T03:25:13.813711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:02.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789276+0000","last_change":"2024-09-12T03:24:31.645766+0000","last_active":"2024-09-12T03:25:14.789276+0000","last_peered":"2024-09-12T03:25:14.789276+0000","last_clean":"2024-09-12T03:25:14.789276+0000","last_became_active":"2024-09-12T03:24:31.644903+0000","last_became_peered":"2024-09-12T03:24:31.644903+0000","last_unstale":"2024-09-12T03:25:14.789276+0000","last_undegraded":"2024-09-12T03:25:14.789276+0000","last_fullsized":"2024-09-12T03:25:14.789276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813333+0000","last_change":"2024-09-12T03:25:13.813333+0000","last_active":"2024-09-12T03:25:13.813333+0000","last_peered":"2024-09-12T03:25:13.813333+0000","last_clean":"2024-09-12T03:25:13.813333+0000","last_became_active":"2024-09-12T03:25:13.812963+0000","last_became_peered":"2024-09-12T03:25:13.812963+0000","last_unstale":"2024-09-12T03:25:13.813333+0000","last_undegraded":"2024-09-12T03:25:13.813333+0000","last_fullsized":"2024-09-12T03:25:13.813333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:38.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.266892+0000","last_change":"2024-09-12T03:25:14.266892+0000","last_active":"2024-09-12T03:25:14.266892+0000","last_peered":"2024-09-12T03:25:14.266892+0000","last_clean":"2024-09-12T03:25:14.266892+0000","last_became_active":"2024-09-12T03:25:14.266452+0000","last_became_peered":"2024-09-12T03:25:14.266452+0000","last_unstale":"2024-09-12T03:25:14.266892+0000","last_undegraded":"2024-09-12T03:25:14.266892+0000","last_fullsized":"2024-09-12T03:25:14.266892+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:58.206639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886393+0000","last_change":"2024-09-12T03:24:31.646199+0000","last_active":"2024-09-12T03:25:13.886393+0000","last_peered":"2024-09-12T03:25:13.886393+0000","last_clean":"2024-09-12T03:25:13.886393+0000","last_became_active":"2024-09-12T03:24:31.645872+0000","last_became_peered":"2024-09-12T03:24:31.645872+0000","last_unstale":"2024-09-12T03:25:13.886393+0000","last_undegraded":"2024-09-12T03:25:13.886393+0000","last_fullsized":"2024-09-12T03:25:13.886393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:21.538730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788752+0000","last_change":"2024-09-12T03:24:31.643704+0000","last_active":"2024-09-12T03:25:14.788752+0000","last_peered":"2024-09-12T03:25:14.788752+0000","last_clean":"2024-09-12T03:25:14.788752+0000","last_became_active":"2024-09-12T03:24:31.643190+0000","last_became_peered":"2024-09-12T03:24:31.643190+0000","last_unstale":"2024-09-12T03:25:14.788752+0000","last_undegraded":"2024-09-12T03:25:14.788752+0000","last_fullsized":"2024-09-12T03:25:14.788752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:23.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809313+0000","last_change":"2024-09-12T03:24:31.643282+0000","last_active":"2024-09-12T03:25:13.809313+0000","last_peered":"2024-09-12T03:25:13.809313+0000","last_clean":"2024-09-12T03:25:13.809313+0000","last_became_active":"2024-09-12T03:24:31.642721+0000","last_became_peered":"2024-09-12T03:24:31.642721+0000","last_unstale":"2024-09-12T03:25:13.809313+0000","last_undegraded":"2024-09-12T03:25:13.809313+0000","last_fullsized":"2024-09-12T03:25:13.809313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:52.025426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809100+0000","last_change":"2024-09-12T03:24:31.650266+0000","last_active":"2024-09-12T03:25:13.809100+0000","last_peered":"2024-09-12T03:25:13.809100+0000","last_clean":"2024-09-12T03:25:13.809100+0000","last_became_active":"2024-09-12T03:24:31.649887+0000","last_became_peered":"2024-09-12T03:24:31.649887+0000","last_unstale":"2024-09-12T03:25:13.809100+0000","last_undegraded":"2024-09-12T03:25:13.809100+0000","last_fullsized":"2024-09-12T03:25:13.809100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:05.043430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813392+0000","last_change":"2024-09-12T03:24:31.639381+0000","last_active":"2024-09-12T03:25:13.813392+0000","last_peered":"2024-09-12T03:25:13.813392+0000","last_clean":"2024-09-12T03:25:13.813392+0000","last_became_active":"2024-09-12T03:24:31.639047+0000","last_became_peered":"2024-09-12T03:24:31.639047+0000","last_unstale":"2024-09-12T03:25:13.813392+0000","last_undegraded":"2024-09-12T03:25:13.813392+0000","last_fullsized":"2024-09-12T03:25:13.813392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:57.608026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810832+0000","last_change":"2024-09-12T03:24:31.646682+0000","last_active":"2024-09-12T03:25:13.810832+0000","last_peered":"2024-09-12T03:25:13.810832+0000","last_clean":"2024-09-12T03:25:13.810832+0000","last_became_active":"2024-09-12T03:24:31.645065+0000","last_became_peered":"2024-09-12T03:24:31.645065+0000","last_unstale":"2024-09-12T03:25:13.810832+0000","last_undegraded":"2024-09-12T03:25:13.810832+0000","last_fullsized":"2024-09-12T03:25:13.810832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:27.328188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811214+0000","last_change":"2024-09-12T03:24:31.641532+0000","last_active":"2024-09-12T03:25:13.811214+0000","last_peered":"2024-09-12T03:25:13.811214+0000","last_clean":"2024-09-12T03:25:13.811214+0000","last_became_active":"2024-09-12T03:24:31.641164+0000","last_became_peered":"2024-09-12T03:24:31.641164+0000","last_unstale":"2024-09-12T03:25:13.811214+0000","last_undegraded":"2024-09-12T03:25:13.811214+0000","last_fullsized":"2024-09-12T03:25:13.811214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:28.984409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265655+0000","last_change":"2024-09-12T03:24:31.646372+0000","last_active":"2024-09-12T03:25:14.265655+0000","last_peered":"2024-09-12T03:25:14.265655+0000","last_clean":"2024-09-12T03:25:14.265655+0000","last_became_active":"2024-09-12T03:24:31.644636+0000","last_became_peered":"2024-09-12T03:24:31.644636+0000","last_unstale":"2024-09-12T03:25:14.265655+0000","last_undegraded":"2024-09-12T03:25:14.265655+0000","last_fullsized":"2024-09-12T03:25:14.265655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:51.023245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":102,"num_read_kb":104,"num_write":156,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":153095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1716224,"data_stored":1150479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":524198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1654784,"data_stored":1115933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":515278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1111521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":520510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":515472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":570745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":558533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":55557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:25:41.174 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:25:41.553 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:41.830 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:41 smithi050 ceph-mon[30378]: from='client.24760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:25:41.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:41 smithi082 ceph-mon[35046]: from='client.24760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:25:42.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:41 smithi142 ceph-mon[35329]: from='client.24760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:25:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 -- 172.21.15.50:0/3273105086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8550103ff0 msgr2=0x7f85501043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:42.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 --2- 172.21.15.50:0/3273105086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8550103ff0 0x7f85501043d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8544009900 tx=0x7f854402a410 comp rx=0 tx=0).stop 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 -- 172.21.15.50:0/3273105086 shutdown_connections 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 --2- 172.21.15.50:0/3273105086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550109120 0x7f8550110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 --2- 172.21.15.50:0/3273105086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85501049a0 0x7f85501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 --2- 172.21.15.50:0/3273105086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8550103ff0 0x7f85501043d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 -- 172.21.15.50:0/3273105086 >> 172.21.15.50:0/3273105086 conn(0x7f85500ffa90 msgr2=0x7f8550101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 -- 172.21.15.50:0/3273105086 shutdown_connections 2024-09-12T03:25:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.085+0000 7f8557400640 1 -- 172.21.15.50:0/3273105086 wait complete. 2024-09-12T03:25:42.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.086+0000 7f8557400640 1 Processor -- start 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 -- start start 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 0x7f855019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85501049a0 0x7f855019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550109120 0x7f855019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8550114170 con 0x7f85501049a0 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8550113ff0 con 0x7f8550109120 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8557400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85501142f0 con 0x7f8550103ff0 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8554a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 0x7f855019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f854fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85501049a0 0x7f855019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8554a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 0x7f855019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:46152/0 (socket says 172.21.15.50:46152) 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.087+0000 7f8554a00640 1 -- 172.21.15.50:0/1709385571 learned_addr learned my addr 172.21.15.50:0/1709385571 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8555400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550109120 0x7f855019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:42.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8554a00640 1 -- 172.21.15.50:0/1709385571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550109120 msgr2=0x7f855019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8554a00640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550109120 0x7f855019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8554a00640 1 -- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85501049a0 msgr2=0x7f855019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8554a00640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85501049a0 0x7f855019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8554a00640 1 -- 172.21.15.50:0/1709385571 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855019f640 con 0x7f8550103ff0 2024-09-12T03:25:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8555400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550109120 0x7f855019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:42.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f854fe00640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85501049a0 0x7f855019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.088+0000 7f8554a00640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 0x7f855019a690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8544004770 tx=0x7f8544003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.089+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854403f070 con 0x7f8550103ff0 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.089+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8544031ca0 con 0x7f8550103ff0 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.089+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855019f8d0 con 0x7f8550103ff0 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.089+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544035030 con 0x7f8550103ff0 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.089+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85501a7210 con 0x7f8550103ff0 2024-09-12T03:25:42.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.090+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85501043d0 con 0x7f8550103ff0 2024-09-12T03:25:42.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.092+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8544042070 con 0x7f8550103ff0 2024-09-12T03:25:42.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.093+0000 7f854d600640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f85380a2c90 0x7f85380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:42.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.093+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f85440f0230 con 0x7f8550103ff0 2024-09-12T03:25:42.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.094+0000 7f854fe00640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f85380a2c90 0x7f85380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:42.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.094+0000 7f854fe00640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f85380a2c90 0x7f85380a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85500fffb0 tx=0x7f853c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:42.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.095+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85440bcc70 con 0x7f8550103ff0 2024-09-12T03:25:42.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.278+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8550062630 con 0x7f85380a2c90 2024-09-12T03:25:42.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.287+0000 7f854d600640 1 -- 172.21.15.50:0/1709385571 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340356 (secure 0 0 0) 0x7f8550062630 con 0x7f85380a2c90 2024-09-12T03:25:42.291 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:42.292 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-12T03:25:42.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f85380a2c90 msgr2=0x7f85380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f85380a2c90 0x7f85380a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85500fffb0 tx=0x7f853c005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 msgr2=0x7f855019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 0x7f855019a690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8544004770 tx=0x7f8544003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 shutdown_connections 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f85380a2c90 0x7f85380a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550109120 0x7f855019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f85501049a0 0x7f855019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 --2- 172.21.15.50:0/1709385571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8550103ff0 0x7f855019a690 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.291+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 >> 172.21.15.50:0/1709385571 conn(0x7f85500ffa90 msgr2=0x7f8550100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.292+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 shutdown_connections 2024-09-12T03:25:42.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:42.292+0000 7f8557400640 1 -- 172.21.15.50:0/1709385571 wait complete. 2024-09-12T03:25:42.788 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:42 smithi050 ceph-mon[30378]: pgmap v336: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-12T03:25:42.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:42 smithi082 ceph-mon[35046]: pgmap v336: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-12T03:25:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:42 smithi142 ceph-mon[35329]: pgmap v336: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-12T03:25:43.144 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-12T03:25:40.761556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649001,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":259,"num_read_kb":244,"num_write":344,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376040,"kb_used_data":14312,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565784,"statfs":{"total":1151940427776,"available":1151555362816,"internally_reserved":0,"allocated":14655488,"data_stored":8161325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51179,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":92,"num_write":32,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.1f","version":"92'9","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.863741+0000","last_change":"2024-09-12T03:24:29.302864+0000","last_active":"2024-09-12T03:25:21.863741+0000","last_peered":"2024-09-12T03:25:21.863741+0000","last_clean":"2024-09-12T03:25:21.863741+0000","last_became_active":"2024-09-12T03:24:29.302551+0000","last_became_peered":"2024-09-12T03:24:29.302551+0000","last_unstale":"2024-09-12T03:25:21.863741+0000","last_undegraded":"2024-09-12T03:25:21.863741+0000","last_fullsized":"2024-09-12T03:25:21.863741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:10.151772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.887020+0000","last_change":"2024-09-12T03:24:31.649557+0000","last_active":"2024-09-12T03:25:13.887020+0000","last_peered":"2024-09-12T03:25:13.887020+0000","last_clean":"2024-09-12T03:25:13.887020+0000","last_became_active":"2024-09-12T03:24:31.649082+0000","last_became_peered":"2024-09-12T03:24:31.649082+0000","last_unstale":"2024-09-12T03:25:13.887020+0000","last_undegraded":"2024-09-12T03:25:13.887020+0000","last_fullsized":"2024-09-12T03:25:13.887020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:54.245063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810710+0000","last_change":"2024-09-12T03:24:40.708670+0000","last_active":"2024-09-12T03:25:13.810710+0000","last_peered":"2024-09-12T03:25:13.810710+0000","last_clean":"2024-09-12T03:25:13.810710+0000","last_became_active":"2024-09-12T03:24:40.708264+0000","last_became_peered":"2024-09-12T03:24:40.708264+0000","last_unstale":"2024-09-12T03:25:13.810710+0000","last_undegraded":"2024-09-12T03:25:13.810710+0000","last_fullsized":"2024-09-12T03:25:13.810710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:00.670492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.887210+0000","last_change":"2024-09-12T03:24:29.312254+0000","last_active":"2024-09-12T03:25:13.887210+0000","last_peered":"2024-09-12T03:25:13.887210+0000","last_clean":"2024-09-12T03:25:13.887210+0000","last_became_active":"2024-09-12T03:24:29.311842+0000","last_became_peered":"2024-09-12T03:24:29.311842+0000","last_unstale":"2024-09-12T03:25:13.887210+0000","last_undegraded":"2024-09-12T03:25:13.887210+0000","last_fullsized":"2024-09-12T03:25:13.887210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:43.298456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811346+0000","last_change":"2024-09-12T03:24:31.644330+0000","last_active":"2024-09-12T03:25:13.811346+0000","last_peered":"2024-09-12T03:25:13.811346+0000","last_clean":"2024-09-12T03:25:13.811346+0000","last_became_active":"2024-09-12T03:24:31.642904+0000","last_became_peered":"2024-09-12T03:24:31.642904+0000","last_unstale":"2024-09-12T03:25:13.811346+0000","last_undegraded":"2024-09-12T03:25:13.811346+0000","last_fullsized":"2024-09-12T03:25:13.811346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:14.304333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812283+0000","last_change":"2024-09-12T03:24:40.706682+0000","last_active":"2024-09-12T03:25:13.812283+0000","last_peered":"2024-09-12T03:25:13.812283+0000","last_clean":"2024-09-12T03:25:13.812283+0000","last_became_active":"2024-09-12T03:24:40.706143+0000","last_became_peered":"2024-09-12T03:24:40.706143+0000","last_unstale":"2024-09-12T03:25:13.812283+0000","last_undegraded":"2024-09-12T03:25:13.812283+0000","last_fullsized":"2024-09-12T03:25:13.812283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:05.146486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.862104+0000","last_change":"2024-09-12T03:24:29.307621+0000","last_active":"2024-09-12T03:25:21.862104+0000","last_peered":"2024-09-12T03:25:21.862104+0000","last_clean":"2024-09-12T03:25:21.862104+0000","last_became_active":"2024-09-12T03:24:29.307258+0000","last_became_peered":"2024-09-12T03:24:29.307258+0000","last_unstale":"2024-09-12T03:25:21.862104+0000","last_undegraded":"2024-09-12T03:25:21.862104+0000","last_fullsized":"2024-09-12T03:25:21.862104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:03.513367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815361+0000","last_change":"2024-09-12T03:24:31.649972+0000","last_active":"2024-09-12T03:25:13.815361+0000","last_peered":"2024-09-12T03:25:13.815361+0000","last_clean":"2024-09-12T03:25:13.815361+0000","last_became_active":"2024-09-12T03:24:31.649693+0000","last_became_peered":"2024-09-12T03:24:31.649693+0000","last_unstale":"2024-09-12T03:25:13.815361+0000","last_undegraded":"2024-09-12T03:25:13.815361+0000","last_fullsized":"2024-09-12T03:25:13.815361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:10.502054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813448+0000","last_change":"2024-09-12T03:24:40.698995+0000","last_active":"2024-09-12T03:25:13.813448+0000","last_peered":"2024-09-12T03:25:13.813448+0000","last_clean":"2024-09-12T03:25:13.813448+0000","last_became_active":"2024-09-12T03:24:40.698094+0000","last_became_peered":"2024-09-12T03:24:40.698094+0000","last_unstale":"2024-09-12T03:25:13.813448+0000","last_undegraded":"2024-09-12T03:25:13.813448+0000","last_fullsized":"2024-09-12T03:25:13.813448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:39.266209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809719+0000","last_change":"2024-09-12T03:24:29.304522+0000","last_active":"2024-09-12T03:25:13.809719+0000","last_peered":"2024-09-12T03:25:13.809719+0000","last_clean":"2024-09-12T03:25:13.809719+0000","last_became_active":"2024-09-12T03:24:29.304206+0000","last_became_peered":"2024-09-12T03:24:29.304206+0000","last_unstale":"2024-09-12T03:25:13.809719+0000","last_undegraded":"2024-09-12T03:25:13.809719+0000","last_fullsized":"2024-09-12T03:25:13.809719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:12.365720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320244+0000","last_change":"2024-09-12T03:24:31.648735+0000","last_active":"2024-09-12T03:25:14.320244+0000","last_peered":"2024-09-12T03:25:14.320244+0000","last_clean":"2024-09-12T03:25:14.320244+0000","last_became_active":"2024-09-12T03:24:31.648416+0000","last_became_peered":"2024-09-12T03:24:31.648416+0000","last_unstale":"2024-09-12T03:25:14.320244+0000","last_undegraded":"2024-09-12T03:25:14.320244+0000","last_fullsized":"2024-09-12T03:25:14.320244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:24.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813389+0000","last_change":"2024-09-12T03:24:40.699941+0000","last_active":"2024-09-12T03:25:13.813389+0000","last_peered":"2024-09-12T03:25:13.813389+0000","last_clean":"2024-09-12T03:25:13.813389+0000","last_became_active":"2024-09-12T03:24:40.699126+0000","last_became_peered":"2024-09-12T03:24:40.699126+0000","last_unstale":"2024-09-12T03:25:13.813389+0000","last_undegraded":"2024-09-12T03:25:13.813389+0000","last_fullsized":"2024-09-12T03:25:13.813389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:24.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788924+0000","last_change":"2024-09-12T03:24:31.650650+0000","last_active":"2024-09-12T03:25:14.788924+0000","last_peered":"2024-09-12T03:25:14.788924+0000","last_clean":"2024-09-12T03:25:14.788924+0000","last_became_active":"2024-09-12T03:24:31.650279+0000","last_became_peered":"2024-09-12T03:24:31.650279+0000","last_unstale":"2024-09-12T03:25:14.788924+0000","last_undegraded":"2024-09-12T03:25:14.788924+0000","last_fullsized":"2024-09-12T03:25:14.788924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:49.860317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857601+0000","last_change":"2024-09-12T03:24:29.307426+0000","last_active":"2024-09-12T03:25:20.857601+0000","last_peered":"2024-09-12T03:25:20.857601+0000","last_clean":"2024-09-12T03:25:20.857601+0000","last_became_active":"2024-09-12T03:24:29.307119+0000","last_became_peered":"2024-09-12T03:24:29.307119+0000","last_unstale":"2024-09-12T03:25:20.857601+0000","last_undegraded":"2024-09-12T03:25:20.857601+0000","last_fullsized":"2024-09-12T03:25:20.857601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:48.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812724+0000","last_change":"2024-09-12T03:24:40.700292+0000","last_active":"2024-09-12T03:25:13.812724+0000","last_peered":"2024-09-12T03:25:13.812724+0000","last_clean":"2024-09-12T03:25:13.812724+0000","last_became_active":"2024-09-12T03:24:40.699819+0000","last_became_peered":"2024-09-12T03:24:40.699819+0000","last_unstale":"2024-09-12T03:25:13.812724+0000","last_undegraded":"2024-09-12T03:25:13.812724+0000","last_fullsized":"2024-09-12T03:25:13.812724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:43.022322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886544+0000","last_change":"2024-09-12T03:24:31.647669+0000","last_active":"2024-09-12T03:25:13.886544+0000","last_peered":"2024-09-12T03:25:13.886544+0000","last_clean":"2024-09-12T03:25:13.886544+0000","last_became_active":"2024-09-12T03:24:31.644429+0000","last_became_peered":"2024-09-12T03:24:31.644429+0000","last_unstale":"2024-09-12T03:25:13.886544+0000","last_undegraded":"2024-09-12T03:25:13.886544+0000","last_fullsized":"2024-09-12T03:25:13.886544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:22.556399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811172+0000","last_change":"2024-09-12T03:24:29.309334+0000","last_active":"2024-09-12T03:25:13.811172+0000","last_peered":"2024-09-12T03:25:13.811172+0000","last_clean":"2024-09-12T03:25:13.811172+0000","last_became_active":"2024-09-12T03:24:29.309007+0000","last_became_peered":"2024-09-12T03:24:29.309007+0000","last_unstale":"2024-09-12T03:25:13.811172+0000","last_undegraded":"2024-09-12T03:25:13.811172+0000","last_fullsized":"2024-09-12T03:25:13.811172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:40.898104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812855+0000","last_change":"2024-09-12T03:24:40.706611+0000","last_active":"2024-09-12T03:25:13.812855+0000","last_peered":"2024-09-12T03:25:13.812855+0000","last_clean":"2024-09-12T03:25:13.812855+0000","last_became_active":"2024-09-12T03:24:40.706058+0000","last_became_peered":"2024-09-12T03:24:40.706058+0000","last_unstale":"2024-09-12T03:25:13.812855+0000","last_undegraded":"2024-09-12T03:25:13.812855+0000","last_fullsized":"2024-09-12T03:25:13.812855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:16.547894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265106+0000","last_change":"2024-09-12T03:24:31.648458+0000","last_active":"2024-09-12T03:25:14.265106+0000","last_peered":"2024-09-12T03:25:14.265106+0000","last_clean":"2024-09-12T03:25:14.265106+0000","last_became_active":"2024-09-12T03:24:31.644171+0000","last_became_peered":"2024-09-12T03:24:31.644171+0000","last_unstale":"2024-09-12T03:25:14.265106+0000","last_undegraded":"2024-09-12T03:25:14.265106+0000","last_fullsized":"2024-09-12T03:25:14.265106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:20.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.858013+0000","last_change":"2024-09-12T03:24:29.309013+0000","last_active":"2024-09-12T03:25:20.858013+0000","last_peered":"2024-09-12T03:25:20.858013+0000","last_clean":"2024-09-12T03:25:20.858013+0000","last_became_active":"2024-09-12T03:24:29.308459+0000","last_became_peered":"2024-09-12T03:24:29.308459+0000","last_unstale":"2024-09-12T03:25:20.858013+0000","last_undegraded":"2024-09-12T03:25:20.858013+0000","last_fullsized":"2024-09-12T03:25:20.858013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:38.655615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.321811+0000","last_change":"2024-09-12T03:25:14.321079+0000","last_active":"2024-09-12T03:25:14.321811+0000","last_peered":"2024-09-12T03:25:14.321811+0000","last_clean":"2024-09-12T03:25:14.321811+0000","last_became_active":"2024-09-12T03:25:14.320602+0000","last_became_peered":"2024-09-12T03:25:14.320602+0000","last_unstale":"2024-09-12T03:25:14.321811+0000","last_undegraded":"2024-09-12T03:25:14.321811+0000","last_fullsized":"2024-09-12T03:25:14.321811+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:18.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809174+0000","last_change":"2024-09-12T03:24:31.637560+0000","last_active":"2024-09-12T03:25:13.809174+0000","last_peered":"2024-09-12T03:25:13.809174+0000","last_clean":"2024-09-12T03:25:13.809174+0000","last_became_active":"2024-09-12T03:24:31.636962+0000","last_became_peered":"2024-09-12T03:24:31.636962+0000","last_unstale":"2024-09-12T03:25:13.809174+0000","last_undegraded":"2024-09-12T03:25:13.809174+0000","last_fullsized":"2024-09-12T03:25:13.809174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:44.468397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857965+0000","last_change":"2024-09-12T03:24:29.305262+0000","last_active":"2024-09-12T03:25:20.857965+0000","last_peered":"2024-09-12T03:25:20.857965+0000","last_clean":"2024-09-12T03:25:20.857965+0000","last_became_active":"2024-09-12T03:24:29.304937+0000","last_became_peered":"2024-09-12T03:24:29.304937+0000","last_unstale":"2024-09-12T03:25:20.857965+0000","last_undegraded":"2024-09-12T03:25:20.857965+0000","last_fullsized":"2024-09-12T03:25:20.857965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:45.172526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812951+0000","last_change":"2024-09-12T03:24:40.702517+0000","last_active":"2024-09-12T03:25:13.812951+0000","last_peered":"2024-09-12T03:25:13.812951+0000","last_clean":"2024-09-12T03:25:13.812951+0000","last_became_active":"2024-09-12T03:24:40.702093+0000","last_became_peered":"2024-09-12T03:24:40.702093+0000","last_unstale":"2024-09-12T03:25:13.812951+0000","last_undegraded":"2024-09-12T03:25:13.812951+0000","last_fullsized":"2024-09-12T03:25:13.812951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:00.817214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811275+0000","last_change":"2024-09-12T03:24:31.650541+0000","last_active":"2024-09-12T03:25:13.811275+0000","last_peered":"2024-09-12T03:25:13.811275+0000","last_clean":"2024-09-12T03:25:13.811275+0000","last_became_active":"2024-09-12T03:24:31.650180+0000","last_became_peered":"2024-09-12T03:24:31.650180+0000","last_unstale":"2024-09-12T03:25:13.811275+0000","last_undegraded":"2024-09-12T03:25:13.811275+0000","last_fullsized":"2024-09-12T03:25:13.811275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:11.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857786+0000","last_change":"2024-09-12T03:24:29.335888+0000","last_active":"2024-09-12T03:25:20.857786+0000","last_peered":"2024-09-12T03:25:20.857786+0000","last_clean":"2024-09-12T03:25:20.857786+0000","last_became_active":"2024-09-12T03:24:29.335535+0000","last_became_peered":"2024-09-12T03:24:29.335535+0000","last_unstale":"2024-09-12T03:25:20.857786+0000","last_undegraded":"2024-09-12T03:25:20.857786+0000","last_fullsized":"2024-09-12T03:25:20.857786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:19.246874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811072+0000","last_change":"2024-09-12T03:24:40.696385+0000","last_active":"2024-09-12T03:25:13.811072+0000","last_peered":"2024-09-12T03:25:13.811072+0000","last_clean":"2024-09-12T03:25:13.811072+0000","last_became_active":"2024-09-12T03:24:40.695968+0000","last_became_peered":"2024-09-12T03:24:40.695968+0000","last_unstale":"2024-09-12T03:25:13.811072+0000","last_undegraded":"2024-09-12T03:25:13.811072+0000","last_fullsized":"2024-09-12T03:25:13.811072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:31.768769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886617+0000","last_change":"2024-09-12T03:24:31.655017+0000","last_active":"2024-09-12T03:25:13.886617+0000","last_peered":"2024-09-12T03:25:13.886617+0000","last_clean":"2024-09-12T03:25:13.886617+0000","last_became_active":"2024-09-12T03:24:31.654597+0000","last_became_peered":"2024-09-12T03:24:31.654597+0000","last_unstale":"2024-09-12T03:25:13.886617+0000","last_undegraded":"2024-09-12T03:25:13.886617+0000","last_fullsized":"2024-09-12T03:25:13.886617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:13.053238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:35.651055+0000","last_change":"2024-09-12T03:24:29.303991+0000","last_active":"2024-09-12T03:25:35.651055+0000","last_peered":"2024-09-12T03:25:35.651055+0000","last_clean":"2024-09-12T03:25:35.651055+0000","last_became_active":"2024-09-12T03:24:29.303682+0000","last_became_peered":"2024-09-12T03:24:29.303682+0000","last_unstale":"2024-09-12T03:25:35.651055+0000","last_undegraded":"2024-09-12T03:25:35.651055+0000","last_fullsized":"2024-09-12T03:25:35.651055+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:59.403461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886212+0000","last_change":"2024-09-12T03:24:40.704304+0000","last_active":"2024-09-12T03:25:13.886212+0000","last_peered":"2024-09-12T03:25:13.886212+0000","last_clean":"2024-09-12T03:25:13.886212+0000","last_became_active":"2024-09-12T03:24:40.703967+0000","last_became_peered":"2024-09-12T03:24:40.703967+0000","last_unstale":"2024-09-12T03:25:13.886212+0000","last_undegraded":"2024-09-12T03:25:13.886212+0000","last_fullsized":"2024-09-12T03:25:13.886212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:00.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942398+0000","last_change":"2024-09-12T03:24:31.650303+0000","last_active":"2024-09-12T03:25:13.942398+0000","last_peered":"2024-09-12T03:25:13.942398+0000","last_clean":"2024-09-12T03:25:13.942398+0000","last_became_active":"2024-09-12T03:24:31.649223+0000","last_became_peered":"2024-09-12T03:24:31.649223+0000","last_unstale":"2024-09-12T03:25:13.942398+0000","last_undegraded":"2024-09-12T03:25:13.942398+0000","last_fullsized":"2024-09-12T03:25:13.942398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:45.190363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857037+0000","last_change":"2024-09-12T03:24:29.310570+0000","last_active":"2024-09-12T03:25:20.857037+0000","last_peered":"2024-09-12T03:25:20.857037+0000","last_clean":"2024-09-12T03:25:20.857037+0000","last_became_active":"2024-09-12T03:24:29.310249+0000","last_became_peered":"2024-09-12T03:24:29.310249+0000","last_unstale":"2024-09-12T03:25:20.857037+0000","last_undegraded":"2024-09-12T03:25:20.857037+0000","last_fullsized":"2024-09-12T03:25:20.857037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.049566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265053+0000","last_change":"2024-09-12T03:24:40.696716+0000","last_active":"2024-09-12T03:25:14.265053+0000","last_peered":"2024-09-12T03:25:14.265053+0000","last_clean":"2024-09-12T03:25:14.265053+0000","last_became_active":"2024-09-12T03:24:40.696322+0000","last_became_peered":"2024-09-12T03:24:40.696322+0000","last_unstale":"2024-09-12T03:25:14.265053+0000","last_undegraded":"2024-09-12T03:25:14.265053+0000","last_fullsized":"2024-09-12T03:25:14.265053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:31.315393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809413+0000","last_change":"2024-09-12T03:24:31.652360+0000","last_active":"2024-09-12T03:25:13.809413+0000","last_peered":"2024-09-12T03:25:13.809413+0000","last_clean":"2024-09-12T03:25:13.809413+0000","last_became_active":"2024-09-12T03:24:31.652066+0000","last_became_peered":"2024-09-12T03:24:31.652066+0000","last_unstale":"2024-09-12T03:25:13.809413+0000","last_undegraded":"2024-09-12T03:25:13.809413+0000","last_fullsized":"2024-09-12T03:25:13.809413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:54:15.312876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808499+0000","last_change":"2024-09-12T03:24:29.305811+0000","last_active":"2024-09-12T03:25:13.808499+0000","last_peered":"2024-09-12T03:25:13.808499+0000","last_clean":"2024-09-12T03:25:13.808499+0000","last_became_active":"2024-09-12T03:24:29.305531+0000","last_became_peered":"2024-09-12T03:24:29.305531+0000","last_unstale":"2024-09-12T03:25:13.808499+0000","last_undegraded":"2024-09-12T03:25:13.808499+0000","last_fullsized":"2024-09-12T03:25:13.808499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:08.496670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.267079+0000","last_change":"2024-09-12T03:25:14.267079+0000","last_active":"2024-09-12T03:25:14.267079+0000","last_peered":"2024-09-12T03:25:14.267079+0000","last_clean":"2024-09-12T03:25:14.267079+0000","last_became_active":"2024-09-12T03:25:14.266625+0000","last_became_peered":"2024-09-12T03:25:14.266625+0000","last_unstale":"2024-09-12T03:25:14.267079+0000","last_undegraded":"2024-09-12T03:25:14.267079+0000","last_fullsized":"2024-09-12T03:25:14.267079+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812997+0000","last_change":"2024-09-12T03:25:13.812997+0000","last_active":"2024-09-12T03:25:13.812997+0000","last_peered":"2024-09-12T03:25:13.812997+0000","last_clean":"2024-09-12T03:25:13.812997+0000","last_became_active":"2024-09-12T03:25:13.812596+0000","last_became_peered":"2024-09-12T03:25:13.812596+0000","last_unstale":"2024-09-12T03:25:13.812997+0000","last_undegraded":"2024-09-12T03:25:13.812997+0000","last_fullsized":"2024-09-12T03:25:13.812997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:43.228416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789061+0000","last_change":"2024-09-12T03:24:29.301269+0000","last_active":"2024-09-12T03:25:14.789061+0000","last_peered":"2024-09-12T03:25:14.789061+0000","last_clean":"2024-09-12T03:25:14.789061+0000","last_became_active":"2024-09-12T03:24:29.300937+0000","last_became_peered":"2024-09-12T03:24:29.300937+0000","last_unstale":"2024-09-12T03:25:14.789061+0000","last_undegraded":"2024-09-12T03:25:14.789061+0000","last_fullsized":"2024-09-12T03:25:14.789061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:24.453709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942183+0000","last_change":"2024-09-12T03:24:40.708631+0000","last_active":"2024-09-12T03:25:13.942183+0000","last_peered":"2024-09-12T03:25:13.942183+0000","last_clean":"2024-09-12T03:25:13.942183+0000","last_became_active":"2024-09-12T03:24:40.706381+0000","last_became_peered":"2024-09-12T03:24:40.706381+0000","last_unstale":"2024-09-12T03:25:13.942183+0000","last_undegraded":"2024-09-12T03:25:13.942183+0000","last_fullsized":"2024-09-12T03:25:13.942183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:10.945538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319147+0000","last_change":"2024-09-12T03:24:31.652471+0000","last_active":"2024-09-12T03:25:14.319147+0000","last_peered":"2024-09-12T03:25:14.319147+0000","last_clean":"2024-09-12T03:25:14.319147+0000","last_became_active":"2024-09-12T03:24:31.651566+0000","last_became_peered":"2024-09-12T03:24:31.651566+0000","last_unstale":"2024-09-12T03:25:14.319147+0000","last_undegraded":"2024-09-12T03:25:14.319147+0000","last_fullsized":"2024-09-12T03:25:14.319147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:01.238554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:36.887572+0000","last_change":"2024-09-12T03:24:29.306214+0000","last_active":"2024-09-12T03:25:36.887572+0000","last_peered":"2024-09-12T03:25:36.887572+0000","last_clean":"2024-09-12T03:25:36.887572+0000","last_became_active":"2024-09-12T03:24:29.305846+0000","last_became_peered":"2024-09-12T03:24:29.305846+0000","last_unstale":"2024-09-12T03:25:36.887572+0000","last_undegraded":"2024-09-12T03:25:36.887572+0000","last_fullsized":"2024-09-12T03:25:36.887572+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:50.940377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942373+0000","last_change":"2024-09-12T03:24:40.709412+0000","last_active":"2024-09-12T03:25:13.942373+0000","last_peered":"2024-09-12T03:25:13.942373+0000","last_clean":"2024-09-12T03:25:13.942373+0000","last_became_active":"2024-09-12T03:24:40.705106+0000","last_became_peered":"2024-09-12T03:24:40.705106+0000","last_unstale":"2024-09-12T03:25:13.942373+0000","last_undegraded":"2024-09-12T03:25:13.942373+0000","last_fullsized":"2024-09-12T03:25:13.942373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:32.963060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941696+0000","last_change":"2024-09-12T03:24:31.644654+0000","last_active":"2024-09-12T03:25:13.941696+0000","last_peered":"2024-09-12T03:25:13.941696+0000","last_clean":"2024-09-12T03:25:13.941696+0000","last_became_active":"2024-09-12T03:24:31.642477+0000","last_became_peered":"2024-09-12T03:24:31.642477+0000","last_unstale":"2024-09-12T03:25:13.941696+0000","last_undegraded":"2024-09-12T03:25:13.941696+0000","last_fullsized":"2024-09-12T03:25:13.941696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.758784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.862588+0000","last_change":"2024-09-12T03:24:29.306850+0000","last_active":"2024-09-12T03:25:21.862588+0000","last_peered":"2024-09-12T03:25:21.862588+0000","last_clean":"2024-09-12T03:25:21.862588+0000","last_became_active":"2024-09-12T03:24:29.305902+0000","last_became_peered":"2024-09-12T03:24:29.305902+0000","last_unstale":"2024-09-12T03:25:21.862588+0000","last_undegraded":"2024-09-12T03:25:21.862588+0000","last_fullsized":"2024-09-12T03:25:21.862588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:02.564521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320379+0000","last_change":"2024-09-12T03:24:40.700966+0000","last_active":"2024-09-12T03:25:14.320379+0000","last_peered":"2024-09-12T03:25:14.320379+0000","last_clean":"2024-09-12T03:25:14.320379+0000","last_became_active":"2024-09-12T03:24:40.700553+0000","last_became_peered":"2024-09-12T03:24:40.700553+0000","last_unstale":"2024-09-12T03:25:14.320379+0000","last_undegraded":"2024-09-12T03:25:14.320379+0000","last_fullsized":"2024-09-12T03:25:14.320379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:14.855459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886794+0000","last_change":"2024-09-12T03:24:31.653431+0000","last_active":"2024-09-12T03:25:13.886794+0000","last_peered":"2024-09-12T03:25:13.886794+0000","last_clean":"2024-09-12T03:25:13.886794+0000","last_became_active":"2024-09-12T03:24:31.648771+0000","last_became_peered":"2024-09-12T03:24:31.648771+0000","last_unstale":"2024-09-12T03:25:13.886794+0000","last_undegraded":"2024-09-12T03:25:13.886794+0000","last_fullsized":"2024-09-12T03:25:13.886794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:54.052828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:22.866402+0000","last_change":"2024-09-12T03:24:29.305825+0000","last_active":"2024-09-12T03:25:22.866402+0000","last_peered":"2024-09-12T03:25:22.866402+0000","last_clean":"2024-09-12T03:25:22.866402+0000","last_became_active":"2024-09-12T03:24:29.305468+0000","last_became_peered":"2024-09-12T03:24:29.305468+0000","last_unstale":"2024-09-12T03:25:22.866402+0000","last_undegraded":"2024-09-12T03:25:22.866402+0000","last_fullsized":"2024-09-12T03:25:22.866402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:30.899307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":363,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788894+0000","last_change":"2024-09-12T03:23:13.507232+0000","last_active":"2024-09-12T03:25:14.788894+0000","last_peered":"2024-09-12T03:25:14.788894+0000","last_clean":"2024-09-12T03:25:14.788894+0000","last_became_active":"2024-09-12T03:23:13.506683+0000","last_became_peered":"2024-09-12T03:23:13.506683+0000","last_unstale":"2024-09-12T03:25:14.788894+0000","last_undegraded":"2024-09-12T03:25:14.788894+0000","last_fullsized":"2024-09-12T03:25:14.788894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_clean_scrub_stamp":"2024-09-12T03:19:33.594997+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.684658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789197+0000","last_change":"2024-09-12T03:24:40.696812+0000","last_active":"2024-09-12T03:25:14.789197+0000","last_peered":"2024-09-12T03:25:14.789197+0000","last_clean":"2024-09-12T03:25:14.789197+0000","last_became_active":"2024-09-12T03:24:40.695586+0000","last_became_peered":"2024-09-12T03:24:40.695586+0000","last_unstale":"2024-09-12T03:25:14.789197+0000","last_undegraded":"2024-09-12T03:25:14.789197+0000","last_fullsized":"2024-09-12T03:25:14.789197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:10.086113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789290+0000","last_change":"2024-09-12T03:24:31.644517+0000","last_active":"2024-09-12T03:25:14.789290+0000","last_peered":"2024-09-12T03:25:14.789290+0000","last_clean":"2024-09-12T03:25:14.789290+0000","last_became_active":"2024-09-12T03:24:31.643996+0000","last_became_peered":"2024-09-12T03:24:31.643996+0000","last_unstale":"2024-09-12T03:25:14.789290+0000","last_undegraded":"2024-09-12T03:25:14.789290+0000","last_fullsized":"2024-09-12T03:25:14.789290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:57.543397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.862918+0000","last_change":"2024-09-12T03:25:14.266550+0000","last_active":"2024-09-12T03:25:21.862918+0000","last_peered":"2024-09-12T03:25:21.862918+0000","last_clean":"2024-09-12T03:25:21.862918+0000","last_became_active":"2024-09-12T03:25:14.266202+0000","last_became_peered":"2024-09-12T03:25:14.266202+0000","last_unstale":"2024-09-12T03:25:21.862918+0000","last_undegraded":"2024-09-12T03:25:21.862918+0000","last_fullsized":"2024-09-12T03:25:21.862918+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:00.981147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815001+0000","last_change":"2024-09-12T03:24:40.697001+0000","last_active":"2024-09-12T03:25:13.815001+0000","last_peered":"2024-09-12T03:25:13.815001+0000","last_clean":"2024-09-12T03:25:13.815001+0000","last_became_active":"2024-09-12T03:24:40.696091+0000","last_became_peered":"2024-09-12T03:24:40.696091+0000","last_unstale":"2024-09-12T03:25:13.815001+0000","last_undegraded":"2024-09-12T03:25:13.815001+0000","last_fullsized":"2024-09-12T03:25:13.815001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:07.073034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789084+0000","last_change":"2024-09-12T03:24:31.655819+0000","last_active":"2024-09-12T03:25:14.789084+0000","last_peered":"2024-09-12T03:25:14.789084+0000","last_clean":"2024-09-12T03:25:14.789084+0000","last_became_active":"2024-09-12T03:24:31.655417+0000","last_became_peered":"2024-09-12T03:24:31.655417+0000","last_unstale":"2024-09-12T03:25:14.789084+0000","last_undegraded":"2024-09-12T03:25:14.789084+0000","last_fullsized":"2024-09-12T03:25:14.789084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:54.380047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320453+0000","last_change":"2024-09-12T03:24:29.303732+0000","last_active":"2024-09-12T03:25:14.320453+0000","last_peered":"2024-09-12T03:25:14.320453+0000","last_clean":"2024-09-12T03:25:14.320453+0000","last_became_active":"2024-09-12T03:24:29.302314+0000","last_became_peered":"2024-09-12T03:24:29.302314+0000","last_unstale":"2024-09-12T03:25:14.320453+0000","last_undegraded":"2024-09-12T03:25:14.320453+0000","last_fullsized":"2024-09-12T03:25:14.320453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:51.555510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.817846+0000","last_change":"2024-09-12T03:25:13.817846+0000","last_active":"2024-09-12T03:25:13.817846+0000","last_peered":"2024-09-12T03:25:13.817846+0000","last_clean":"2024-09-12T03:25:13.817846+0000","last_became_active":"2024-09-12T03:25:13.817102+0000","last_became_peered":"2024-09-12T03:25:13.817102+0000","last_unstale":"2024-09-12T03:25:13.817846+0000","last_undegraded":"2024-09-12T03:25:13.817846+0000","last_fullsized":"2024-09-12T03:25:13.817846+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:26.472731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789248+0000","last_change":"2024-09-12T03:24:31.645293+0000","last_active":"2024-09-12T03:25:14.789248+0000","last_peered":"2024-09-12T03:25:14.789248+0000","last_clean":"2024-09-12T03:25:14.789248+0000","last_became_active":"2024-09-12T03:24:31.644944+0000","last_became_peered":"2024-09-12T03:24:31.644944+0000","last_unstale":"2024-09-12T03:25:14.789248+0000","last_undegraded":"2024-09-12T03:25:14.789248+0000","last_fullsized":"2024-09-12T03:25:14.789248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:38.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942492+0000","last_change":"2024-09-12T03:24:29.302710+0000","last_active":"2024-09-12T03:25:13.942492+0000","last_peered":"2024-09-12T03:25:13.942492+0000","last_clean":"2024-09-12T03:25:13.942492+0000","last_became_active":"2024-09-12T03:24:29.302262+0000","last_became_peered":"2024-09-12T03:24:29.302262+0000","last_unstale":"2024-09-12T03:25:13.942492+0000","last_undegraded":"2024-09-12T03:25:13.942492+0000","last_fullsized":"2024-09-12T03:25:13.942492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:08.392390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.318966+0000","last_change":"2024-09-12T03:24:40.696164+0000","last_active":"2024-09-12T03:25:14.318966+0000","last_peered":"2024-09-12T03:25:14.318966+0000","last_clean":"2024-09-12T03:25:14.318966+0000","last_became_active":"2024-09-12T03:24:40.695786+0000","last_became_peered":"2024-09-12T03:24:40.695786+0000","last_unstale":"2024-09-12T03:25:14.318966+0000","last_undegraded":"2024-09-12T03:25:14.318966+0000","last_fullsized":"2024-09-12T03:25:14.318966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:27.063196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265366+0000","last_change":"2024-09-12T03:24:31.645341+0000","last_active":"2024-09-12T03:25:14.265366+0000","last_peered":"2024-09-12T03:25:14.265366+0000","last_clean":"2024-09-12T03:25:14.265366+0000","last_became_active":"2024-09-12T03:24:31.644496+0000","last_became_peered":"2024-09-12T03:24:31.644496+0000","last_unstale":"2024-09-12T03:25:14.265366+0000","last_undegraded":"2024-09-12T03:25:14.265366+0000","last_fullsized":"2024-09-12T03:25:14.265366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:09.416806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:22.866391+0000","last_change":"2024-09-12T03:24:29.305762+0000","last_active":"2024-09-12T03:25:22.866391+0000","last_peered":"2024-09-12T03:25:22.866391+0000","last_clean":"2024-09-12T03:25:22.866391+0000","last_became_active":"2024-09-12T03:24:29.305445+0000","last_became_peered":"2024-09-12T03:24:29.305445+0000","last_unstale":"2024-09-12T03:25:22.866391+0000","last_undegraded":"2024-09-12T03:25:22.866391+0000","last_fullsized":"2024-09-12T03:25:22.866391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:50.032960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319111+0000","last_change":"2024-09-12T03:24:40.703795+0000","last_active":"2024-09-12T03:25:14.319111+0000","last_peered":"2024-09-12T03:25:14.319111+0000","last_clean":"2024-09-12T03:25:14.319111+0000","last_became_active":"2024-09-12T03:24:40.703361+0000","last_became_peered":"2024-09-12T03:24:40.703361+0000","last_unstale":"2024-09-12T03:25:14.319111+0000","last_undegraded":"2024-09-12T03:25:14.319111+0000","last_fullsized":"2024-09-12T03:25:14.319111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:12.430431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319348+0000","last_change":"2024-09-12T03:24:31.649521+0000","last_active":"2024-09-12T03:25:14.319348+0000","last_peered":"2024-09-12T03:25:14.319348+0000","last_clean":"2024-09-12T03:25:14.319348+0000","last_became_active":"2024-09-12T03:24:31.649217+0000","last_became_peered":"2024-09-12T03:24:31.649217+0000","last_unstale":"2024-09-12T03:25:14.319348+0000","last_undegraded":"2024-09-12T03:25:14.319348+0000","last_fullsized":"2024-09-12T03:25:14.319348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:33.885153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.865005+0000","last_change":"2024-09-12T03:24:29.308568+0000","last_active":"2024-09-12T03:25:21.865005+0000","last_peered":"2024-09-12T03:25:21.865005+0000","last_clean":"2024-09-12T03:25:21.865005+0000","last_became_active":"2024-09-12T03:24:29.308263+0000","last_became_peered":"2024-09-12T03:24:29.308263+0000","last_unstale":"2024-09-12T03:25:21.865005+0000","last_undegraded":"2024-09-12T03:25:21.865005+0000","last_fullsized":"2024-09-12T03:25:21.865005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:54.383248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942390+0000","last_change":"2024-09-12T03:24:40.707719+0000","last_active":"2024-09-12T03:25:13.942390+0000","last_peered":"2024-09-12T03:25:13.942390+0000","last_clean":"2024-09-12T03:25:13.942390+0000","last_became_active":"2024-09-12T03:24:40.707364+0000","last_became_peered":"2024-09-12T03:24:40.707364+0000","last_unstale":"2024-09-12T03:25:13.942390+0000","last_undegraded":"2024-09-12T03:25:13.942390+0000","last_fullsized":"2024-09-12T03:25:13.942390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:00.423368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788656+0000","last_change":"2024-09-12T03:24:31.646006+0000","last_active":"2024-09-12T03:25:14.788656+0000","last_peered":"2024-09-12T03:25:14.788656+0000","last_clean":"2024-09-12T03:25:14.788656+0000","last_became_active":"2024-09-12T03:24:31.643165+0000","last_became_peered":"2024-09-12T03:24:31.643165+0000","last_unstale":"2024-09-12T03:25:14.788656+0000","last_undegraded":"2024-09-12T03:25:14.788656+0000","last_fullsized":"2024-09-12T03:25:14.788656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:33.883658+0000","last_change":"2024-09-12T03:24:29.304521+0000","last_active":"2024-09-12T03:25:33.883658+0000","last_peered":"2024-09-12T03:25:33.883658+0000","last_clean":"2024-09-12T03:25:33.883658+0000","last_became_active":"2024-09-12T03:24:29.304147+0000","last_became_peered":"2024-09-12T03:24:29.304147+0000","last_unstale":"2024-09-12T03:25:33.883658+0000","last_undegraded":"2024-09-12T03:25:33.883658+0000","last_fullsized":"2024-09-12T03:25:33.883658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:17.834282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808706+0000","last_change":"2024-09-12T03:24:40.706844+0000","last_active":"2024-09-12T03:25:13.808706+0000","last_peered":"2024-09-12T03:25:13.808706+0000","last_clean":"2024-09-12T03:25:13.808706+0000","last_became_active":"2024-09-12T03:24:40.704829+0000","last_became_peered":"2024-09-12T03:24:40.704829+0000","last_unstale":"2024-09-12T03:25:13.808706+0000","last_undegraded":"2024-09-12T03:25:13.808706+0000","last_fullsized":"2024-09-12T03:25:13.808706+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:32.875316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788924+0000","last_change":"2024-09-12T03:24:31.650168+0000","last_active":"2024-09-12T03:25:14.788924+0000","last_peered":"2024-09-12T03:25:14.788924+0000","last_clean":"2024-09-12T03:25:14.788924+0000","last_became_active":"2024-09-12T03:24:31.649754+0000","last_became_peered":"2024-09-12T03:24:31.649754+0000","last_unstale":"2024-09-12T03:25:14.788924+0000","last_undegraded":"2024-09-12T03:25:14.788924+0000","last_fullsized":"2024-09-12T03:25:14.788924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:09.615836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.858421+0000","last_change":"2024-09-12T03:24:29.303488+0000","last_active":"2024-09-12T03:25:20.858421+0000","last_peered":"2024-09-12T03:25:20.858421+0000","last_clean":"2024-09-12T03:25:20.858421+0000","last_became_active":"2024-09-12T03:24:29.301858+0000","last_became_peered":"2024-09-12T03:24:29.301858+0000","last_unstale":"2024-09-12T03:25:20.858421+0000","last_undegraded":"2024-09-12T03:25:20.858421+0000","last_fullsized":"2024-09-12T03:25:20.858421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:27.846820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265096+0000","last_change":"2024-09-12T03:24:40.697734+0000","last_active":"2024-09-12T03:25:14.265096+0000","last_peered":"2024-09-12T03:25:14.265096+0000","last_clean":"2024-09-12T03:25:14.265096+0000","last_became_active":"2024-09-12T03:24:40.697271+0000","last_became_peered":"2024-09-12T03:24:40.697271+0000","last_unstale":"2024-09-12T03:25:14.265096+0000","last_undegraded":"2024-09-12T03:25:14.265096+0000","last_fullsized":"2024-09-12T03:25:14.265096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.336322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941957+0000","last_change":"2024-09-12T03:24:31.647040+0000","last_active":"2024-09-12T03:25:13.941957+0000","last_peered":"2024-09-12T03:25:13.941957+0000","last_clean":"2024-09-12T03:25:13.941957+0000","last_became_active":"2024-09-12T03:24:31.643365+0000","last_became_peered":"2024-09-12T03:24:31.643365+0000","last_unstale":"2024-09-12T03:25:13.941957+0000","last_undegraded":"2024-09-12T03:25:13.941957+0000","last_fullsized":"2024-09-12T03:25:13.941957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:06.638005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.864927+0000","last_change":"2024-09-12T03:24:29.303390+0000","last_active":"2024-09-12T03:25:21.864927+0000","last_peered":"2024-09-12T03:25:21.864927+0000","last_clean":"2024-09-12T03:25:21.864927+0000","last_became_active":"2024-09-12T03:24:29.303072+0000","last_became_peered":"2024-09-12T03:24:29.303072+0000","last_unstale":"2024-09-12T03:25:21.864927+0000","last_undegraded":"2024-09-12T03:25:21.864927+0000","last_fullsized":"2024-09-12T03:25:21.864927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:01.572888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320143+0000","last_change":"2024-09-12T03:24:40.707977+0000","last_active":"2024-09-12T03:25:14.320143+0000","last_peered":"2024-09-12T03:25:14.320143+0000","last_clean":"2024-09-12T03:25:14.320143+0000","last_became_active":"2024-09-12T03:24:40.707497+0000","last_became_peered":"2024-09-12T03:24:40.707497+0000","last_unstale":"2024-09-12T03:25:14.320143+0000","last_undegraded":"2024-09-12T03:25:14.320143+0000","last_fullsized":"2024-09-12T03:25:14.320143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:45.474021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809346+0000","last_change":"2024-09-12T03:24:31.642417+0000","last_active":"2024-09-12T03:25:13.809346+0000","last_peered":"2024-09-12T03:25:13.809346+0000","last_clean":"2024-09-12T03:25:13.809346+0000","last_became_active":"2024-09-12T03:24:31.642076+0000","last_became_peered":"2024-09-12T03:24:31.642076+0000","last_unstale":"2024-09-12T03:25:13.809346+0000","last_undegraded":"2024-09-12T03:25:13.809346+0000","last_fullsized":"2024-09-12T03:25:13.809346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:56.436088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'30","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:38.895152+0000","last_change":"2024-09-12T03:24:29.302982+0000","last_active":"2024-09-12T03:25:38.895152+0000","last_peered":"2024-09-12T03:25:38.895152+0000","last_clean":"2024-09-12T03:25:38.895152+0000","last_became_active":"2024-09-12T03:24:29.302369+0000","last_became_peered":"2024-09-12T03:24:29.302369+0000","last_unstale":"2024-09-12T03:25:38.895152+0000","last_undegraded":"2024-09-12T03:25:38.895152+0000","last_fullsized":"2024-09-12T03:25:38.895152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:24:30.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54396,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":73,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808161+0000","last_change":"2024-09-12T03:24:40.696745+0000","last_active":"2024-09-12T03:25:13.808161+0000","last_peered":"2024-09-12T03:25:13.808161+0000","last_clean":"2024-09-12T03:25:13.808161+0000","last_became_active":"2024-09-12T03:24:40.696273+0000","last_became_peered":"2024-09-12T03:24:40.696273+0000","last_unstale":"2024-09-12T03:25:13.808161+0000","last_undegraded":"2024-09-12T03:25:13.808161+0000","last_fullsized":"2024-09-12T03:25:13.808161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.519765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815432+0000","last_change":"2024-09-12T03:24:31.650567+0000","last_active":"2024-09-12T03:25:13.815432+0000","last_peered":"2024-09-12T03:25:13.815432+0000","last_clean":"2024-09-12T03:25:13.815432+0000","last_became_active":"2024-09-12T03:24:31.650242+0000","last_became_peered":"2024-09-12T03:24:31.650242+0000","last_unstale":"2024-09-12T03:25:13.815432+0000","last_undegraded":"2024-09-12T03:25:13.815432+0000","last_fullsized":"2024-09-12T03:25:13.815432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:14.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.858384+0000","last_change":"2024-09-12T03:24:29.310630+0000","last_active":"2024-09-12T03:25:20.858384+0000","last_peered":"2024-09-12T03:25:20.858384+0000","last_clean":"2024-09-12T03:25:20.858384+0000","last_became_active":"2024-09-12T03:24:29.310302+0000","last_became_peered":"2024-09-12T03:24:29.310302+0000","last_unstale":"2024-09-12T03:25:20.858384+0000","last_undegraded":"2024-09-12T03:25:20.858384+0000","last_fullsized":"2024-09-12T03:25:20.858384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:53.835246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812536+0000","last_change":"2024-09-12T03:24:40.698941+0000","last_active":"2024-09-12T03:25:13.812536+0000","last_peered":"2024-09-12T03:25:13.812536+0000","last_clean":"2024-09-12T03:25:13.812536+0000","last_became_active":"2024-09-12T03:24:40.698323+0000","last_became_peered":"2024-09-12T03:24:40.698323+0000","last_unstale":"2024-09-12T03:25:13.812536+0000","last_undegraded":"2024-09-12T03:25:13.812536+0000","last_fullsized":"2024-09-12T03:25:13.812536+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:29.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.320685+0000","last_change":"2024-09-12T03:25:14.320685+0000","last_active":"2024-09-12T03:25:14.320685+0000","last_peered":"2024-09-12T03:25:14.320685+0000","last_clean":"2024-09-12T03:25:14.320685+0000","last_became_active":"2024-09-12T03:25:14.320295+0000","last_became_peered":"2024-09-12T03:25:14.320295+0000","last_unstale":"2024-09-12T03:25:14.320685+0000","last_undegraded":"2024-09-12T03:25:14.320685+0000","last_fullsized":"2024-09-12T03:25:14.320685+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:42.459136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:20.857188+0000","last_change":"2024-09-12T03:24:29.302988+0000","last_active":"2024-09-12T03:25:20.857188+0000","last_peered":"2024-09-12T03:25:20.857188+0000","last_clean":"2024-09-12T03:25:20.857188+0000","last_became_active":"2024-09-12T03:24:29.302569+0000","last_became_peered":"2024-09-12T03:24:29.302569+0000","last_unstale":"2024-09-12T03:25:20.857188+0000","last_undegraded":"2024-09-12T03:25:20.857188+0000","last_fullsized":"2024-09-12T03:25:20.857188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:00.196284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.807973+0000","last_change":"2024-09-12T03:24:40.701486+0000","last_active":"2024-09-12T03:25:13.807973+0000","last_peered":"2024-09-12T03:25:13.807973+0000","last_clean":"2024-09-12T03:25:13.807973+0000","last_became_active":"2024-09-12T03:24:40.701078+0000","last_became_peered":"2024-09-12T03:24:40.701078+0000","last_unstale":"2024-09-12T03:25:13.807973+0000","last_undegraded":"2024-09-12T03:25:13.807973+0000","last_fullsized":"2024-09-12T03:25:13.807973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:50.032773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815679+0000","last_change":"2024-09-12T03:24:31.646681+0000","last_active":"2024-09-12T03:25:13.815679+0000","last_peered":"2024-09-12T03:25:13.815679+0000","last_clean":"2024-09-12T03:25:13.815679+0000","last_became_active":"2024-09-12T03:24:31.646389+0000","last_became_peered":"2024-09-12T03:24:31.646389+0000","last_unstale":"2024-09-12T03:25:13.815679+0000","last_undegraded":"2024-09-12T03:25:13.815679+0000","last_fullsized":"2024-09-12T03:25:13.815679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:37.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:37.889548+0000","last_change":"2024-09-12T03:24:29.299316+0000","last_active":"2024-09-12T03:25:37.889548+0000","last_peered":"2024-09-12T03:25:37.889548+0000","last_clean":"2024-09-12T03:25:37.889548+0000","last_became_active":"2024-09-12T03:24:29.298900+0000","last_became_peered":"2024-09-12T03:24:29.298900+0000","last_unstale":"2024-09-12T03:25:37.889548+0000","last_undegraded":"2024-09-12T03:25:37.889548+0000","last_fullsized":"2024-09-12T03:25:37.889548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:36.650938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886404+0000","last_change":"2024-09-12T03:24:40.702034+0000","last_active":"2024-09-12T03:25:13.886404+0000","last_peered":"2024-09-12T03:25:13.886404+0000","last_clean":"2024-09-12T03:25:13.886404+0000","last_became_active":"2024-09-12T03:24:40.699312+0000","last_became_peered":"2024-09-12T03:24:40.699312+0000","last_unstale":"2024-09-12T03:25:13.886404+0000","last_undegraded":"2024-09-12T03:25:13.886404+0000","last_fullsized":"2024-09-12T03:25:13.886404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:58.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811172+0000","last_change":"2024-09-12T03:24:40.701744+0000","last_active":"2024-09-12T03:25:13.811172+0000","last_peered":"2024-09-12T03:25:13.811172+0000","last_clean":"2024-09-12T03:25:13.811172+0000","last_became_active":"2024-09-12T03:24:40.700929+0000","last_became_peered":"2024-09-12T03:24:40.700929+0000","last_unstale":"2024-09-12T03:25:13.811172+0000","last_undegraded":"2024-09-12T03:25:13.811172+0000","last_fullsized":"2024-09-12T03:25:13.811172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.851810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808040+0000","last_change":"2024-09-12T03:24:31.647774+0000","last_active":"2024-09-12T03:25:13.808040+0000","last_peered":"2024-09-12T03:25:13.808040+0000","last_clean":"2024-09-12T03:25:13.808040+0000","last_became_active":"2024-09-12T03:24:31.647329+0000","last_became_peered":"2024-09-12T03:24:31.647329+0000","last_unstale":"2024-09-12T03:25:13.808040+0000","last_undegraded":"2024-09-12T03:25:13.808040+0000","last_fullsized":"2024-09-12T03:25:13.808040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:00.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319360+0000","last_change":"2024-09-12T03:24:29.307455+0000","last_active":"2024-09-12T03:25:14.319360+0000","last_peered":"2024-09-12T03:25:14.319360+0000","last_clean":"2024-09-12T03:25:14.319360+0000","last_became_active":"2024-09-12T03:24:29.307120+0000","last_became_peered":"2024-09-12T03:24:29.307120+0000","last_unstale":"2024-09-12T03:25:14.319360+0000","last_undegraded":"2024-09-12T03:25:14.319360+0000","last_fullsized":"2024-09-12T03:25:14.319360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:29.374097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265451+0000","last_change":"2024-09-12T03:24:40.695149+0000","last_active":"2024-09-12T03:25:14.265451+0000","last_peered":"2024-09-12T03:25:14.265451+0000","last_clean":"2024-09-12T03:25:14.265451+0000","last_became_active":"2024-09-12T03:24:40.694755+0000","last_became_peered":"2024-09-12T03:24:40.694755+0000","last_unstale":"2024-09-12T03:25:14.265451+0000","last_undegraded":"2024-09-12T03:25:14.265451+0000","last_fullsized":"2024-09-12T03:25:14.265451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:34.045481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886493+0000","last_change":"2024-09-12T03:24:31.654674+0000","last_active":"2024-09-12T03:25:13.886493+0000","last_peered":"2024-09-12T03:25:13.886493+0000","last_clean":"2024-09-12T03:25:13.886493+0000","last_became_active":"2024-09-12T03:24:31.654319+0000","last_became_peered":"2024-09-12T03:24:31.654319+0000","last_unstale":"2024-09-12T03:25:13.886493+0000","last_undegraded":"2024-09-12T03:25:13.886493+0000","last_fullsized":"2024-09-12T03:25:13.886493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:33.441162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.864549+0000","last_change":"2024-09-12T03:24:29.307849+0000","last_active":"2024-09-12T03:25:21.864549+0000","last_peered":"2024-09-12T03:25:21.864549+0000","last_clean":"2024-09-12T03:25:21.864549+0000","last_became_active":"2024-09-12T03:24:29.306017+0000","last_became_peered":"2024-09-12T03:24:29.306017+0000","last_unstale":"2024-09-12T03:25:21.864549+0000","last_undegraded":"2024-09-12T03:25:21.864549+0000","last_fullsized":"2024-09-12T03:25:21.864549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:37.478289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:21.865430+0000","last_change":"2024-09-12T03:24:29.302410+0000","last_active":"2024-09-12T03:25:21.865430+0000","last_peered":"2024-09-12T03:25:21.865430+0000","last_clean":"2024-09-12T03:25:21.865430+0000","last_became_active":"2024-09-12T03:24:29.302031+0000","last_became_peered":"2024-09-12T03:24:29.302031+0000","last_unstale":"2024-09-12T03:25:21.865430+0000","last_undegraded":"2024-09-12T03:25:21.865430+0000","last_fullsized":"2024-09-12T03:25:21.865430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:11.698690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319976+0000","last_change":"2024-09-12T03:24:31.652823+0000","last_active":"2024-09-12T03:25:14.319976+0000","last_peered":"2024-09-12T03:25:14.319976+0000","last_clean":"2024-09-12T03:25:14.319976+0000","last_became_active":"2024-09-12T03:24:31.652438+0000","last_became_peered":"2024-09-12T03:24:31.652438+0000","last_unstale":"2024-09-12T03:25:14.319976+0000","last_undegraded":"2024-09-12T03:25:14.319976+0000","last_fullsized":"2024-09-12T03:25:14.319976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942339+0000","last_change":"2024-09-12T03:25:13.942339+0000","last_active":"2024-09-12T03:25:13.942339+0000","last_peered":"2024-09-12T03:25:13.942339+0000","last_clean":"2024-09-12T03:25:13.942339+0000","last_became_active":"2024-09-12T03:25:13.942030+0000","last_became_peered":"2024-09-12T03:25:13.942030+0000","last_unstale":"2024-09-12T03:25:13.942339+0000","last_undegraded":"2024-09-12T03:25:13.942339+0000","last_fullsized":"2024-09-12T03:25:13.942339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:39.584996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:37.890268+0000","last_change":"2024-09-12T03:24:29.303318+0000","last_active":"2024-09-12T03:25:37.890268+0000","last_peered":"2024-09-12T03:25:37.890268+0000","last_clean":"2024-09-12T03:25:37.890268+0000","last_became_active":"2024-09-12T03:24:29.303001+0000","last_became_peered":"2024-09-12T03:24:29.303001+0000","last_unstale":"2024-09-12T03:25:37.890268+0000","last_undegraded":"2024-09-12T03:25:37.890268+0000","last_fullsized":"2024-09-12T03:25:37.890268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:08.094697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810264+0000","last_change":"2024-09-12T03:24:31.644465+0000","last_active":"2024-09-12T03:25:13.810264+0000","last_peered":"2024-09-12T03:25:13.810264+0000","last_clean":"2024-09-12T03:25:13.810264+0000","last_became_active":"2024-09-12T03:24:31.644141+0000","last_became_peered":"2024-09-12T03:24:31.644141+0000","last_unstale":"2024-09-12T03:25:13.810264+0000","last_undegraded":"2024-09-12T03:25:13.810264+0000","last_fullsized":"2024-09-12T03:25:13.810264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:37.448614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811203+0000","last_change":"2024-09-12T03:24:40.695190+0000","last_active":"2024-09-12T03:25:13.811203+0000","last_peered":"2024-09-12T03:25:13.811203+0000","last_clean":"2024-09-12T03:25:13.811203+0000","last_became_active":"2024-09-12T03:24:40.694746+0000","last_became_peered":"2024-09-12T03:24:40.694746+0000","last_unstale":"2024-09-12T03:25:13.811203+0000","last_undegraded":"2024-09-12T03:25:13.811203+0000","last_fullsized":"2024-09-12T03:25:13.811203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:31.905437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265069+0000","last_change":"2024-09-12T03:24:31.649068+0000","last_active":"2024-09-12T03:25:14.265069+0000","last_peered":"2024-09-12T03:25:14.265069+0000","last_clean":"2024-09-12T03:25:14.265069+0000","last_became_active":"2024-09-12T03:24:31.648761+0000","last_became_peered":"2024-09-12T03:24:31.648761+0000","last_unstale":"2024-09-12T03:25:14.265069+0000","last_undegraded":"2024-09-12T03:25:14.265069+0000","last_fullsized":"2024-09-12T03:25:14.265069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:08.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811318+0000","last_change":"2024-09-12T03:24:31.643604+0000","last_active":"2024-09-12T03:25:13.811318+0000","last_peered":"2024-09-12T03:25:13.811318+0000","last_clean":"2024-09-12T03:25:13.811318+0000","last_became_active":"2024-09-12T03:24:31.643053+0000","last_became_peered":"2024-09-12T03:24:31.643053+0000","last_unstale":"2024-09-12T03:25:13.811318+0000","last_undegraded":"2024-09-12T03:25:13.811318+0000","last_fullsized":"2024-09-12T03:25:13.811318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:59.814891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810842+0000","last_change":"2024-09-12T03:24:31.648376+0000","last_active":"2024-09-12T03:25:13.810842+0000","last_peered":"2024-09-12T03:25:13.810842+0000","last_clean":"2024-09-12T03:25:13.810842+0000","last_became_active":"2024-09-12T03:24:31.648076+0000","last_became_peered":"2024-09-12T03:24:31.648076+0000","last_unstale":"2024-09-12T03:25:13.810842+0000","last_undegraded":"2024-09-12T03:25:13.810842+0000","last_fullsized":"2024-09-12T03:25:13.810842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:47.736362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810502+0000","last_change":"2024-09-12T03:24:31.652489+0000","last_active":"2024-09-12T03:25:13.810502+0000","last_peered":"2024-09-12T03:25:13.810502+0000","last_clean":"2024-09-12T03:25:13.810502+0000","last_became_active":"2024-09-12T03:24:31.651907+0000","last_became_peered":"2024-09-12T03:24:31.651907+0000","last_unstale":"2024-09-12T03:25:13.810502+0000","last_undegraded":"2024-09-12T03:25:13.810502+0000","last_fullsized":"2024-09-12T03:25:13.810502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:24.584264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.942284+0000","last_change":"2024-09-12T03:24:31.649967+0000","last_active":"2024-09-12T03:25:13.942284+0000","last_peered":"2024-09-12T03:25:13.942284+0000","last_clean":"2024-09-12T03:25:13.942284+0000","last_became_active":"2024-09-12T03:24:31.649687+0000","last_became_peered":"2024-09-12T03:24:31.649687+0000","last_unstale":"2024-09-12T03:25:13.942284+0000","last_undegraded":"2024-09-12T03:25:13.942284+0000","last_fullsized":"2024-09-12T03:25:13.942284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:14.875352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265499+0000","last_change":"2024-09-12T03:24:31.645115+0000","last_active":"2024-09-12T03:25:14.265499+0000","last_peered":"2024-09-12T03:25:14.265499+0000","last_clean":"2024-09-12T03:25:14.265499+0000","last_became_active":"2024-09-12T03:24:31.644442+0000","last_became_peered":"2024-09-12T03:24:31.644442+0000","last_unstale":"2024-09-12T03:25:14.265499+0000","last_undegraded":"2024-09-12T03:25:14.265499+0000","last_fullsized":"2024-09-12T03:25:14.265499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:28.539948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.812088+0000","last_change":"2024-09-12T03:24:31.643283+0000","last_active":"2024-09-12T03:25:13.812088+0000","last_peered":"2024-09-12T03:25:13.812088+0000","last_clean":"2024-09-12T03:25:13.812088+0000","last_became_active":"2024-09-12T03:24:31.642935+0000","last_became_peered":"2024-09-12T03:24:31.642935+0000","last_unstale":"2024-09-12T03:25:13.812088+0000","last_undegraded":"2024-09-12T03:25:13.812088+0000","last_fullsized":"2024-09-12T03:25:13.812088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:30.754875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.817610+0000","last_change":"2024-09-12T03:25:13.817610+0000","last_active":"2024-09-12T03:25:13.817610+0000","last_peered":"2024-09-12T03:25:13.817610+0000","last_clean":"2024-09-12T03:25:13.817610+0000","last_became_active":"2024-09-12T03:25:13.817008+0000","last_became_peered":"2024-09-12T03:25:13.817008+0000","last_unstale":"2024-09-12T03:25:13.817610+0000","last_undegraded":"2024-09-12T03:25:13.817610+0000","last_fullsized":"2024-09-12T03:25:13.817610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:45.334353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.319122+0000","last_change":"2024-09-12T03:24:31.656588+0000","last_active":"2024-09-12T03:25:14.319122+0000","last_peered":"2024-09-12T03:25:14.319122+0000","last_clean":"2024-09-12T03:25:14.319122+0000","last_became_active":"2024-09-12T03:24:31.656294+0000","last_became_peered":"2024-09-12T03:24:31.656294+0000","last_unstale":"2024-09-12T03:25:14.319122+0000","last_undegraded":"2024-09-12T03:25:14.319122+0000","last_fullsized":"2024-09-12T03:25:14.319122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.855802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886506+0000","last_change":"2024-09-12T03:24:31.648986+0000","last_active":"2024-09-12T03:25:13.886506+0000","last_peered":"2024-09-12T03:25:13.886506+0000","last_clean":"2024-09-12T03:25:13.886506+0000","last_became_active":"2024-09-12T03:24:31.648144+0000","last_became_peered":"2024-09-12T03:24:31.648144+0000","last_unstale":"2024-09-12T03:25:13.886506+0000","last_undegraded":"2024-09-12T03:25:13.886506+0000","last_fullsized":"2024-09-12T03:25:13.886506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.823742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.815183+0000","last_change":"2024-09-12T03:24:31.644390+0000","last_active":"2024-09-12T03:25:13.815183+0000","last_peered":"2024-09-12T03:25:13.815183+0000","last_clean":"2024-09-12T03:25:13.815183+0000","last_became_active":"2024-09-12T03:24:31.642660+0000","last_became_peered":"2024-09-12T03:24:31.642660+0000","last_unstale":"2024-09-12T03:25:13.815183+0000","last_undegraded":"2024-09-12T03:25:13.815183+0000","last_fullsized":"2024-09-12T03:25:13.815183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:58.577883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941756+0000","last_change":"2024-09-12T03:24:31.649895+0000","last_active":"2024-09-12T03:25:13.941756+0000","last_peered":"2024-09-12T03:25:13.941756+0000","last_clean":"2024-09-12T03:25:13.941756+0000","last_became_active":"2024-09-12T03:24:31.649141+0000","last_became_peered":"2024-09-12T03:24:31.649141+0000","last_unstale":"2024-09-12T03:25:13.941756+0000","last_undegraded":"2024-09-12T03:25:13.941756+0000","last_fullsized":"2024-09-12T03:25:13.941756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:48.175496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265330+0000","last_change":"2024-09-12T03:24:31.640402+0000","last_active":"2024-09-12T03:25:14.265330+0000","last_peered":"2024-09-12T03:25:14.265330+0000","last_clean":"2024-09-12T03:25:14.265330+0000","last_became_active":"2024-09-12T03:24:31.639987+0000","last_became_peered":"2024-09-12T03:24:31.639987+0000","last_unstale":"2024-09-12T03:25:14.265330+0000","last_undegraded":"2024-09-12T03:25:14.265330+0000","last_fullsized":"2024-09-12T03:25:14.265330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:51.540149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811506+0000","last_change":"2024-09-12T03:24:31.647524+0000","last_active":"2024-09-12T03:25:13.811506+0000","last_peered":"2024-09-12T03:25:13.811506+0000","last_clean":"2024-09-12T03:25:13.811506+0000","last_became_active":"2024-09-12T03:24:31.647204+0000","last_became_peered":"2024-09-12T03:24:31.647204+0000","last_unstale":"2024-09-12T03:25:13.811506+0000","last_undegraded":"2024-09-12T03:25:13.811506+0000","last_fullsized":"2024-09-12T03:25:13.811506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.895586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.808709+0000","last_change":"2024-09-12T03:24:31.640797+0000","last_active":"2024-09-12T03:25:13.808709+0000","last_peered":"2024-09-12T03:25:13.808709+0000","last_clean":"2024-09-12T03:25:13.808709+0000","last_became_active":"2024-09-12T03:24:31.640479+0000","last_became_peered":"2024-09-12T03:24:31.640479+0000","last_unstale":"2024-09-12T03:25:13.808709+0000","last_undegraded":"2024-09-12T03:25:13.808709+0000","last_fullsized":"2024-09-12T03:25:13.808709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:08.762939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265755+0000","last_change":"2024-09-12T03:24:31.644225+0000","last_active":"2024-09-12T03:25:14.265755+0000","last_peered":"2024-09-12T03:25:14.265755+0000","last_clean":"2024-09-12T03:25:14.265755+0000","last_became_active":"2024-09-12T03:24:31.643877+0000","last_became_peered":"2024-09-12T03:24:31.643877+0000","last_unstale":"2024-09-12T03:25:14.265755+0000","last_undegraded":"2024-09-12T03:25:14.265755+0000","last_fullsized":"2024-09-12T03:25:14.265755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:50.511392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.814692+0000","last_change":"2024-09-12T03:24:31.644917+0000","last_active":"2024-09-12T03:25:13.814692+0000","last_peered":"2024-09-12T03:25:13.814692+0000","last_clean":"2024-09-12T03:25:13.814692+0000","last_became_active":"2024-09-12T03:24:31.644591+0000","last_became_peered":"2024-09-12T03:24:31.644591+0000","last_unstale":"2024-09-12T03:25:13.814692+0000","last_undegraded":"2024-09-12T03:25:13.814692+0000","last_fullsized":"2024-09-12T03:25:13.814692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:21.961959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810723+0000","last_change":"2024-09-12T03:24:31.648146+0000","last_active":"2024-09-12T03:25:13.810723+0000","last_peered":"2024-09-12T03:25:13.810723+0000","last_clean":"2024-09-12T03:25:13.810723+0000","last_became_active":"2024-09-12T03:24:31.641423+0000","last_became_peered":"2024-09-12T03:24:31.641423+0000","last_unstale":"2024-09-12T03:25:13.810723+0000","last_undegraded":"2024-09-12T03:25:13.810723+0000","last_fullsized":"2024-09-12T03:25:13.810723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:50.985126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811003+0000","last_change":"2024-09-12T03:24:31.641030+0000","last_active":"2024-09-12T03:25:13.811003+0000","last_peered":"2024-09-12T03:25:13.811003+0000","last_clean":"2024-09-12T03:25:13.811003+0000","last_became_active":"2024-09-12T03:24:31.639015+0000","last_became_peered":"2024-09-12T03:24:31.639015+0000","last_unstale":"2024-09-12T03:25:13.811003+0000","last_undegraded":"2024-09-12T03:25:13.811003+0000","last_fullsized":"2024-09-12T03:25:13.811003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.961304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.941758+0000","last_change":"2024-09-12T03:24:31.644601+0000","last_active":"2024-09-12T03:25:13.941758+0000","last_peered":"2024-09-12T03:25:13.941758+0000","last_clean":"2024-09-12T03:25:13.941758+0000","last_became_active":"2024-09-12T03:24:31.644122+0000","last_became_peered":"2024-09-12T03:24:31.644122+0000","last_unstale":"2024-09-12T03:25:13.941758+0000","last_undegraded":"2024-09-12T03:25:13.941758+0000","last_fullsized":"2024-09-12T03:25:13.941758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:06.335464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813711+0000","last_change":"2024-09-12T03:24:31.641496+0000","last_active":"2024-09-12T03:25:13.813711+0000","last_peered":"2024-09-12T03:25:13.813711+0000","last_clean":"2024-09-12T03:25:13.813711+0000","last_became_active":"2024-09-12T03:24:31.641195+0000","last_became_peered":"2024-09-12T03:24:31.641195+0000","last_unstale":"2024-09-12T03:25:13.813711+0000","last_undegraded":"2024-09-12T03:25:13.813711+0000","last_fullsized":"2024-09-12T03:25:13.813711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:02.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.789276+0000","last_change":"2024-09-12T03:24:31.645766+0000","last_active":"2024-09-12T03:25:14.789276+0000","last_peered":"2024-09-12T03:25:14.789276+0000","last_clean":"2024-09-12T03:25:14.789276+0000","last_became_active":"2024-09-12T03:24:31.644903+0000","last_became_peered":"2024-09-12T03:24:31.644903+0000","last_unstale":"2024-09-12T03:25:14.789276+0000","last_undegraded":"2024-09-12T03:25:14.789276+0000","last_fullsized":"2024-09-12T03:25:14.789276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813333+0000","last_change":"2024-09-12T03:25:13.813333+0000","last_active":"2024-09-12T03:25:13.813333+0000","last_peered":"2024-09-12T03:25:13.813333+0000","last_clean":"2024-09-12T03:25:13.813333+0000","last_became_active":"2024-09-12T03:25:13.812963+0000","last_became_peered":"2024-09-12T03:25:13.812963+0000","last_unstale":"2024-09-12T03:25:13.813333+0000","last_undegraded":"2024-09-12T03:25:13.813333+0000","last_fullsized":"2024-09-12T03:25:13.813333+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:38.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.266892+0000","last_change":"2024-09-12T03:25:14.266892+0000","last_active":"2024-09-12T03:25:14.266892+0000","last_peered":"2024-09-12T03:25:14.266892+0000","last_clean":"2024-09-12T03:25:14.266892+0000","last_became_active":"2024-09-12T03:25:14.266452+0000","last_became_peered":"2024-09-12T03:25:14.266452+0000","last_unstale":"2024-09-12T03:25:14.266892+0000","last_undegraded":"2024-09-12T03:25:14.266892+0000","last_fullsized":"2024-09-12T03:25:14.266892+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:58.206639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.886393+0000","last_change":"2024-09-12T03:24:31.646199+0000","last_active":"2024-09-12T03:25:13.886393+0000","last_peered":"2024-09-12T03:25:13.886393+0000","last_clean":"2024-09-12T03:25:13.886393+0000","last_became_active":"2024-09-12T03:24:31.645872+0000","last_became_peered":"2024-09-12T03:24:31.645872+0000","last_unstale":"2024-09-12T03:25:13.886393+0000","last_undegraded":"2024-09-12T03:25:13.886393+0000","last_fullsized":"2024-09-12T03:25:13.886393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:21.538730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.788752+0000","last_change":"2024-09-12T03:24:31.643704+0000","last_active":"2024-09-12T03:25:14.788752+0000","last_peered":"2024-09-12T03:25:14.788752+0000","last_clean":"2024-09-12T03:25:14.788752+0000","last_became_active":"2024-09-12T03:24:31.643190+0000","last_became_peered":"2024-09-12T03:24:31.643190+0000","last_unstale":"2024-09-12T03:25:14.788752+0000","last_undegraded":"2024-09-12T03:25:14.788752+0000","last_fullsized":"2024-09-12T03:25:14.788752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:23.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809313+0000","last_change":"2024-09-12T03:24:31.643282+0000","last_active":"2024-09-12T03:25:13.809313+0000","last_peered":"2024-09-12T03:25:13.809313+0000","last_clean":"2024-09-12T03:25:13.809313+0000","last_became_active":"2024-09-12T03:24:31.642721+0000","last_became_peered":"2024-09-12T03:24:31.642721+0000","last_unstale":"2024-09-12T03:25:13.809313+0000","last_undegraded":"2024-09-12T03:25:13.809313+0000","last_fullsized":"2024-09-12T03:25:13.809313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:52.025426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.809100+0000","last_change":"2024-09-12T03:24:31.650266+0000","last_active":"2024-09-12T03:25:13.809100+0000","last_peered":"2024-09-12T03:25:13.809100+0000","last_clean":"2024-09-12T03:25:13.809100+0000","last_became_active":"2024-09-12T03:24:31.649887+0000","last_became_peered":"2024-09-12T03:24:31.649887+0000","last_unstale":"2024-09-12T03:25:13.809100+0000","last_undegraded":"2024-09-12T03:25:13.809100+0000","last_fullsized":"2024-09-12T03:25:13.809100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:05.043430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.813392+0000","last_change":"2024-09-12T03:24:31.639381+0000","last_active":"2024-09-12T03:25:13.813392+0000","last_peered":"2024-09-12T03:25:13.813392+0000","last_clean":"2024-09-12T03:25:13.813392+0000","last_became_active":"2024-09-12T03:24:31.639047+0000","last_became_peered":"2024-09-12T03:24:31.639047+0000","last_unstale":"2024-09-12T03:25:13.813392+0000","last_undegraded":"2024-09-12T03:25:13.813392+0000","last_fullsized":"2024-09-12T03:25:13.813392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:57.608026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.810832+0000","last_change":"2024-09-12T03:24:31.646682+0000","last_active":"2024-09-12T03:25:13.810832+0000","last_peered":"2024-09-12T03:25:13.810832+0000","last_clean":"2024-09-12T03:25:13.810832+0000","last_became_active":"2024-09-12T03:24:31.645065+0000","last_became_peered":"2024-09-12T03:24:31.645065+0000","last_unstale":"2024-09-12T03:25:13.810832+0000","last_undegraded":"2024-09-12T03:25:13.810832+0000","last_fullsized":"2024-09-12T03:25:13.810832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:27.328188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:13.811214+0000","last_change":"2024-09-12T03:24:31.641532+0000","last_active":"2024-09-12T03:25:13.811214+0000","last_peered":"2024-09-12T03:25:13.811214+0000","last_clean":"2024-09-12T03:25:13.811214+0000","last_became_active":"2024-09-12T03:24:31.641164+0000","last_became_peered":"2024-09-12T03:24:31.641164+0000","last_unstale":"2024-09-12T03:25:13.811214+0000","last_undegraded":"2024-09-12T03:25:13.811214+0000","last_fullsized":"2024-09-12T03:25:13.811214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:28.984409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T03:25:14.265655+0000","last_change":"2024-09-12T03:24:31.646372+0000","last_active":"2024-09-12T03:25:14.265655+0000","last_peered":"2024-09-12T03:25:14.265655+0000","last_clean":"2024-09-12T03:25:14.265655+0000","last_became_active":"2024-09-12T03:24:31.644636+0000","last_became_peered":"2024-09-12T03:24:31.644636+0000","last_unstale":"2024-09-12T03:25:14.265655+0000","last_undegraded":"2024-09-12T03:25:14.265655+0000","last_fullsized":"2024-09-12T03:25:14.265655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:51.023245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58399,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":123,"num_read_kb":125,"num_write":161,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710752,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1728512,"data_stored":1161530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":524198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1654784,"data_stored":1115933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":515278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1111521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":520510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":515472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":570745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":569584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":55557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":55974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:25:43.149 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:25:43.150 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T03:25:43.150 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T03:25:43.527 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:43.835 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:43 smithi050 ceph-mon[30378]: from='client.24766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:25:43.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:43 smithi082 ceph-mon[35046]: from='client.24766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:25:44.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:43 smithi142 ceph-mon[35329]: from='client.24766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:25:44.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 -- 172.21.15.50:0/572126530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c1037c0 msgr2=0x7f2e3c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:44.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 --2- 172.21.15.50:0/572126530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c1037c0 0x7f2e3c10a050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009ca0 tx=0x7f2e3402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 -- 172.21.15.50:0/572126530 shutdown_connections 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 --2- 172.21.15.50:0/572126530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c1037c0 0x7f2e3c10a050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 --2- 172.21.15.50:0/572126530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 0x7f2e3c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 --2- 172.21.15.50:0/572126530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e3c101c00 0x7f2e3c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 -- 172.21.15.50:0/572126530 >> 172.21.15.50:0/572126530 conn(0x7f2e3c0fd390 msgr2=0x7f2e3c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 -- 172.21.15.50:0/572126530 shutdown_connections 2024-09-12T03:25:44.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.232+0000 7f2e41c00640 1 -- 172.21.15.50:0/572126530 wait complete. 2024-09-12T03:25:44.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.233+0000 7f2e41c00640 1 Processor -- start 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.233+0000 7f2e41c00640 1 -- start start 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e41c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 0x7f2e3c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e41c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 0x7f2e3c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e41c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e3c1037c0 0x7f2e3c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e41c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c10d8d0 con 0x7f2e3c101c00 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e41c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c10d750 con 0x7f2e3c1037c0 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e41c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c10da50 con 0x7f2e3c102e00 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 0x7f2e3c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 0x7f2e3c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 0x7f2e3c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57078/0 (socket says 172.21.15.50:57078) 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3aa00640 1 -- 172.21.15.50:0/2655424620 learned_addr learned my addr 172.21.15.50:0/2655424620 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:44.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3b400640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e3c1037c0 0x7f2e3c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3a000640 1 -- 172.21.15.50:0/2655424620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e3c1037c0 msgr2=0x7f2e3c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3a000640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e3c1037c0 0x7f2e3c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3a000640 1 -- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 msgr2=0x7f2e3c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3a000640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 0x7f2e3c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.234+0000 7f2e3a000640 1 -- 172.21.15.50:0/2655424620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c0753e0 con 0x7f2e3c102e00 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e3aa00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 0x7f2e3c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e3a000640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 0x7f2e3c0745e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800e9f0 tx=0x7f2e2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800ccc0 con 0x7f2e3c102e00 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e28005020 con 0x7f2e3c102e00 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c1a1fa0 con 0x7f2e3c102e00 2024-09-12T03:25:44.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280049e0 con 0x7f2e3c102e00 2024-09-12T03:25:44.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.235+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c1a2400 con 0x7f2e3c102e00 2024-09-12T03:25:44.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.236+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c108f30 con 0x7f2e3c102e00 2024-09-12T03:25:44.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.239+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2e28005560 con 0x7f2e3c102e00 2024-09-12T03:25:44.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.240+0000 7f2e33400640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2e200a2c10 0x7f2e200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:44.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.240+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2e280cf4d0 con 0x7f2e3c102e00 2024-09-12T03:25:44.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.241+0000 7f2e3aa00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2e200a2c10 0x7f2e200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:44.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.241+0000 7f2e3aa00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2e200a2c10 0x7f2e200a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0058c0 tx=0x7f2e2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:44.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.244+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e280051d0 con 0x7f2e3c102e00 2024-09-12T03:25:44.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.496+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2e3c02da10 con 0x7f2e3c102e00 2024-09-12T03:25:44.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.503+0000 7f2e33400640 1 -- 172.21.15.50:0/2655424620 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2e2809bf10 con 0x7f2e3c102e00 2024-09-12T03:25:44.504 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-09-12T03:25:44.504 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQA4X+JmgvbEHRAA0tW0szvWDMW8ce8RU8oOag== 2024-09-12T03:25:44.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.505+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2e200a2c10 msgr2=0x7f2e200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:44.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.505+0000 7f2e41c00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2e200a2c10 0x7f2e200a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0058c0 tx=0x7f2e2c005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:44.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.505+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 msgr2=0x7f2e3c0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.505+0000 7f2e41c00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 0x7f2e3c0745e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800e9f0 tx=0x7f2e2800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 shutdown_connections 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2e200a2c10 0x7f2e200a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e3c1037c0 0x7f2e3c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e3c102e00 0x7f2e3c0745e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 --2- 172.21.15.50:0/2655424620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e3c101c00 0x7f2e3c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 >> 172.21.15.50:0/2655424620 conn(0x7f2e3c0fd390 msgr2=0x7f2e3c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 shutdown_connections 2024-09-12T03:25:44.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:44.506+0000 7f2e41c00640 1 -- 172.21.15.50:0/2655424620 wait complete. 2024-09-12T03:25:44.850 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:44 smithi050 ceph-mon[30378]: pgmap v337: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:25:44.850 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:44 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2655424620' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:25:44.850 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:44 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:25:44.850 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:44 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:25:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:44 smithi082 ceph-mon[35046]: pgmap v337: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:25:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:44 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2655424620' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:25:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:44 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:25:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:44 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:25:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:44 smithi142 ceph-mon[35329]: pgmap v337: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:25:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:44 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2655424620' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:25:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:44 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:25:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:44 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:25:45.239 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:25:45.240 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T03:25:45.240 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:25:45.322 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:25:45.322 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:25:45.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T03:25:45.737 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:46.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 -- 172.21.15.50:0/1344048962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7238103ff0 msgr2=0x7f72381043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 --2- 172.21.15.50:0/1344048962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7238103ff0 0x7f72381043d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7228009900 tx=0x7f722802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 -- 172.21.15.50:0/1344048962 shutdown_connections 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 --2- 172.21.15.50:0/1344048962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238109120 0x7f7238110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 --2- 172.21.15.50:0/1344048962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72381049a0 0x7f72381089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 --2- 172.21.15.50:0/1344048962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7238103ff0 0x7f72381043d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 -- 172.21.15.50:0/1344048962 >> 172.21.15.50:0/1344048962 conn(0x7f72380ffa90 msgr2=0x7f7238101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 -- 172.21.15.50:0/1344048962 shutdown_connections 2024-09-12T03:25:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.284+0000 7f723e000640 1 -- 172.21.15.50:0/1344048962 wait complete. 2024-09-12T03:25:46.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.285+0000 7f723e000640 1 Processor -- start 2024-09-12T03:25:46.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.285+0000 7f723e000640 1 -- start start 2024-09-12T03:25:46.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f723e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238103ff0 0x7f723819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f723e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 0x7f723819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f723e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7238109120 0x7f723819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f723e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7238114190 con 0x7f72381049a0 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f723e000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7238114010 con 0x7f7238103ff0 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f723e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7238114310 con 0x7f7238109120 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7236a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 0x7f723819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7237400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238103ff0 0x7f723819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7236a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 0x7f723819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57104/0 (socket says 172.21.15.50:57104) 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7236a00640 1 -- 172.21.15.50:0/944731853 learned_addr learned my addr 172.21.15.50:0/944731853 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7237400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238103ff0 0x7f723819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:37558/0 (socket says 172.21.15.50:37558) 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7237e00640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7238109120 0x7f723819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.286+0000 7f7236a00640 1 -- 172.21.15.50:0/944731853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7238109120 msgr2=0x7f723819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f7236a00640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7238109120 0x7f723819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f7236a00640 1 -- 172.21.15.50:0/944731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238103ff0 msgr2=0x7f723819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f7236a00640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238103ff0 0x7f723819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f7236a00640 1 -- 172.21.15.50:0/944731853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723819f640 con 0x7f72381049a0 2024-09-12T03:25:46.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f7236a00640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 0x7f723819abd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f722400e9f0 tx=0x7f722400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:46.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722400cd80 con 0x7f72381049a0 2024-09-12T03:25:46.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7224005500 con 0x7f72381049a0 2024-09-12T03:25:46.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723819f930 con 0x7f72381049a0 2024-09-12T03:25:46.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224010690 con 0x7f72381049a0 2024-09-12T03:25:46.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.287+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72381a7210 con 0x7f72381049a0 2024-09-12T03:25:46.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.288+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72381043d0 con 0x7f72381049a0 2024-09-12T03:25:46.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.291+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f72240046c0 con 0x7f72381049a0 2024-09-12T03:25:46.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.292+0000 7f722fe00640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f721c0a2c10 0x7f721c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:46.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.293+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f72240cf9e0 con 0x7f72381049a0 2024-09-12T03:25:46.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.293+0000 7f7237400640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f721c0a2c10 0x7f721c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:46.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.293+0000 7f7237400640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f721c0a2c10 0x7f721c0a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7228000c00 tx=0x7f7228005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:46.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.297+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f722409c420 con 0x7f72381049a0 2024-09-12T03:25:46.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.532+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f723810bd70 con 0x7f72381049a0 2024-09-12T03:25:46.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.541+0000 7f722fe00640 1 -- 172.21.15.50:0/944731853 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263194 (secure 0 0 0) 0x7f722409bd20 con 0x7f72381049a0 2024-09-12T03:25:46.545 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:46.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.544+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f721c0a2c10 msgr2=0x7f721c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.544+0000 7f723e000640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f721c0a2c10 0x7f721c0a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7228000c00 tx=0x7f7228005b50 comp rx=0 tx=0).stop 2024-09-12T03:25:46.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.544+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 msgr2=0x7f723819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:46.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.544+0000 7f723e000640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 0x7f723819abd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f722400e9f0 tx=0x7f722400edf0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 shutdown_connections 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f721c0a2c10 0x7f721c0a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7238109120 0x7f723819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72381049a0 0x7f723819abd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 --2- 172.21.15.50:0/944731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7238103ff0 0x7f723819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 >> 172.21.15.50:0/944731853 conn(0x7f72380ffa90 msgr2=0x7f7238100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 shutdown_connections 2024-09-12T03:25:46.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:46.545+0000 7f723e000640 1 -- 172.21.15.50:0/944731853 wait complete. 2024-09-12T03:25:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:46 smithi082 ceph-mon[35046]: pgmap v338: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T03:25:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:46 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/944731853' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:25:46.898 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:46 smithi050 ceph-mon[30378]: pgmap v338: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T03:25:46.898 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:46 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/944731853' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:25:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:46 smithi142 ceph-mon[35329]: pgmap v338: 129 pgs: 129 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T03:25:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:46 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/944731853' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:25:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":63385930},{"type":"v1","addr":"172.21.15.50:6801","nonce":63385930}]},"active_addr":"172.21.15.50:6801/63385930","active_change":"2024-09-12T03:17:12.653136+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.50:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3692206583}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3889049657}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1993742081}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":308265429}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":2536882194}]}]} 2024-09-12T03:25:47.286 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T03:25:47.287 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T03:25:47.287 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:25:47.664 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:48.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.194+0000 7facaa600640 1 -- 172.21.15.50:0/1069481580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4109140 msgr2=0x7faca4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:48.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.194+0000 7facaa600640 1 --2- 172.21.15.50:0/1069481580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4109140 0x7faca4110cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009ca0 tx=0x7fac9c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 -- 172.21.15.50:0/1069481580 shutdown_connections 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 --2- 172.21.15.50:0/1069481580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4109140 0x7faca4110cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 --2- 172.21.15.50:0/1069481580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca41049c0 0x7faca4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 --2- 172.21.15.50:0/1069481580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca4104010 0x7faca41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 -- 172.21.15.50:0/1069481580 >> 172.21.15.50:0/1069481580 conn(0x7faca40ffa50 msgr2=0x7faca4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 -- 172.21.15.50:0/1069481580 shutdown_connections 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.195+0000 7facaa600640 1 -- 172.21.15.50:0/1069481580 wait complete. 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 Processor -- start 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 -- start start 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4104010 0x7faca4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca41049c0 0x7faca4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 0x7faca4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faca41141c0 con 0x7faca4104010 2024-09-12T03:25:48.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faca4114040 con 0x7faca4109140 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7facaa600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faca4114340 con 0x7faca41049c0 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.196+0000 7faca3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4104010 0x7faca4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 0x7faca4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 0x7faca4077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:37570/0 (socket says 172.21.15.50:37570) 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7fac9be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca41049c0 0x7faca4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 -- 172.21.15.50:0/659351822 learned_addr learned my addr 172.21.15.50:0/659351822 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 -- 172.21.15.50:0/659351822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca41049c0 msgr2=0x7faca4076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca41049c0 0x7faca4076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 -- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4104010 msgr2=0x7faca4076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4104010 0x7faca4076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 -- 172.21.15.50:0/659351822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca41aad00 con 0x7faca4109140 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.197+0000 7faca3e00640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 0x7faca4077180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac9c031a80 tx=0x7fac9c031ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:48.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.198+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003b70 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.198+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c003d10 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.198+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca41aaf90 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.198+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c02fcc0 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.198+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41ab4f0 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.199+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca41054f0 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.201+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac9c004b40 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.202+0000 7faca0c00640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac880a2c90 0x7fac880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.202+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fac9c0f06c0 con 0x7faca4109140 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.202+0000 7faca3400640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac880a2c90 0x7fac880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.203+0000 7faca3400640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac880a2c90 0x7fac880a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac94009990 tx=0x7fac94009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:48.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.204+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac9c0bd2a0 con 0x7faca4109140 2024-09-12T03:25:48.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.389+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faca4077c30 con 0x7faca4109140 2024-09-12T03:25:48.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.391+0000 7faca0c00640 1 -- 172.21.15.50:0/659351822 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24622 (secure 0 0 0) 0x7fac9c0bcba0 con 0x7faca4109140 2024-09-12T03:25:48.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:48.392 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":92,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:25:13.804593+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac880a2c90 msgr2=0x7fac880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac880a2c90 0x7fac880a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac94009990 tx=0x7fac94009110 comp rx=0 tx=0).stop 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 msgr2=0x7faca4077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 0x7faca4077180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac9c031a80 tx=0x7fac9c031ab0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 shutdown_connections 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac880a2c90 0x7fac880a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faca4109140 0x7faca4077180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca41049c0 0x7faca4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 --2- 172.21.15.50:0/659351822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca4104010 0x7faca4076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 >> 172.21.15.50:0/659351822 conn(0x7faca40ffa50 msgr2=0x7faca4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:48.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 shutdown_connections 2024-09-12T03:25:48.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:48.395+0000 7facaa600640 1 -- 172.21.15.50:0/659351822 wait complete. 2024-09-12T03:25:48.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:48 smithi050 ceph-mon[30378]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T03:25:48.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:48 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/659351822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:48 smithi082 ceph-mon[35046]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T03:25:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:48 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/659351822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:48 smithi142 ceph-mon[35329]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T03:25:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:48 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/659351822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:49.108 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T03:25:49.108 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:25:49.491 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 -- 172.21.15.50:0/2173474490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 msgr2=0x7fa680074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 --2- 172.21.15.50:0/2173474490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 0x7fa680074970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa670009860 tx=0x7fa67002a3d0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 -- 172.21.15.50:0/2173474490 shutdown_connections 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 --2- 172.21.15.50:0/2173474490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa680114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 --2- 172.21.15.50:0/2173474490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680074eb0 0x7fa680112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 --2- 172.21.15.50:0/2173474490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 0x7fa680074970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.011+0000 7fa686600640 1 -- 172.21.15.50:0/2173474490 >> 172.21.15.50:0/2173474490 conn(0x7fa6800fd8c0 msgr2=0x7fa6800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.012+0000 7fa686600640 1 -- 172.21.15.50:0/2173474490 shutdown_connections 2024-09-12T03:25:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.012+0000 7fa686600640 1 -- 172.21.15.50:0/2173474490 wait complete. 2024-09-12T03:25:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.013+0000 7fa686600640 1 Processor -- start 2024-09-12T03:25:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.013+0000 7fa686600640 1 -- start start 2024-09-12T03:25:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.013+0000 7fa686600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 0x7fa6801a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.013+0000 7fa686600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680074eb0 0x7fa6801a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa686600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa68019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa686600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801185a0 con 0x7fa680112b90 2024-09-12T03:25:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa686600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa680118420 con 0x7fa680074eb0 2024-09-12T03:25:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa686600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa680118720 con 0x7fa680074590 2024-09-12T03:25:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa68019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa68019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48782/0 (socket says 172.21.15.50:48782) 2024-09-12T03:25:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 -- 172.21.15.50:0/69405731 learned_addr learned my addr 172.21.15.50:0/69405731 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67f400640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 0x7fa6801a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67ea00640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680074eb0 0x7fa6801a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 -- 172.21.15.50:0/69405731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 msgr2=0x7fa6801a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 0x7fa6801a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 -- 172.21.15.50:0/69405731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680074eb0 msgr2=0x7fa6801a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680074eb0 0x7fa6801a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.014+0000 7fa67fe00640 1 -- 172.21.15.50:0/69405731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68019f000 con 0x7fa680112b90 2024-09-12T03:25:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.015+0000 7fa67fe00640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa68019e8d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c3c0 tx=0x7fa67800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.015+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67801b070 con 0x7fa680112b90 2024-09-12T03:25:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.015+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68019f2f0 con 0x7fa680112b90 2024-09-12T03:25:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.015+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa67800bb10 con 0x7fa680112b90 2024-09-12T03:25:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.015+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678016d80 con 0x7fa680112b90 2024-09-12T03:25:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.015+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801ab5e0 con 0x7fa680112b90 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.016+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa680074970 con 0x7fa680112b90 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.018+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa67801e070 con 0x7fa680112b90 2024-09-12T03:25:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.019+0000 7fa677e00640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6640a2c90 0x7fa6640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.020+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa6780cf8a0 con 0x7fa680112b90 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.020+0000 7fa67f400640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6640a2c90 0x7fa6640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.021+0000 7fa67f400640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6640a2c90 0x7fa6640a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa67002a9c0 tx=0x7fa670005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.022+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa67809bc90 con 0x7fa680112b90 2024-09-12T03:25:50.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.208+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa68019fe00 con 0x7fa680112b90 2024-09-12T03:25:50.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.210+0000 7fa677e00640 1 -- 172.21.15.50:0/69405731 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24622 (secure 0 0 0) 0x7fa68019fe00 con 0x7fa680112b90 2024-09-12T03:25:50.211 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:25:50.212 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":92,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:25:13.804593+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:25:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6640a2c90 msgr2=0x7fa6640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6640a2c90 0x7fa6640a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa67002a9c0 tx=0x7fa670005c90 comp rx=0 tx=0).stop 2024-09-12T03:25:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 msgr2=0x7fa68019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa68019e8d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c3c0 tx=0x7fa67800c880 comp rx=0 tx=0).stop 2024-09-12T03:25:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 shutdown_connections 2024-09-12T03:25:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6640a2c90 0x7fa6640a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680112b90 0x7fa68019e8d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680074eb0 0x7fa6801a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 --2- 172.21.15.50:0/69405731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa680074590 0x7fa6801a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.212+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 >> 172.21.15.50:0/69405731 conn(0x7fa6800fd8c0 msgr2=0x7fa6800fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.213+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 shutdown_connections 2024-09-12T03:25:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:50.213+0000 7fa686600640 1 -- 172.21.15.50:0/69405731 wait complete. 2024-09-12T03:25:50.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:50 smithi082 ceph-mon[35046]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:25:50.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:50 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/69405731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:50.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:50 smithi050 ceph-mon[30378]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:25:50.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:50 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/69405731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:51.026 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T03:25:51.026 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T03:25:51.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T03:25:51.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T03:25:51.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T03:25:51.028 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T03:25:51.028 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T03:25:51.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T03:25:51.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T03:25:51.030 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T03:25:51.030 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T03:25:51.031 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T03:25:51.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:50 smithi142 ceph-mon[35329]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:25:51.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:50 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/69405731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:25:52.005 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.135 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.711 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:52 smithi050 ceph-mon[30378]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:25:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 -- 172.21.15.50:0/1393664452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec44103ff0 msgr2=0x7fec441043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1393664452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec44103ff0 0x7fec441043d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec34009900 tx=0x7fec3402a410 comp rx=0 tx=0).stop 2024-09-12T03:25:52.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 -- 172.21.15.50:0/1393664452 shutdown_connections 2024-09-12T03:25:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1393664452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec44109120 0x7fec44110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1393664452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec441049a0 0x7fec441089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1393664452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec44103ff0 0x7fec441043d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 -- 172.21.15.50:0/1393664452 >> 172.21.15.50:0/1393664452 conn(0x7fec440ffa90 msgr2=0x7fec44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.824+0000 7fec4aa00640 1 -- 172.21.15.50:0/1393664452 shutdown_connections 2024-09-12T03:25:52.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.825+0000 7fec4aa00640 1 -- 172.21.15.50:0/1393664452 wait complete. 2024-09-12T03:25:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.825+0000 7fec4aa00640 1 Processor -- start 2024-09-12T03:25:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 -- start start 2024-09-12T03:25:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec44103ff0 0x7fec4419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec441049a0 0x7fec4419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec44109120 0x7fec4419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec441141a0 con 0x7fec441049a0 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec44114020 con 0x7fec44103ff0 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec4aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec44114320 con 0x7fec44109120 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec43e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec44103ff0 0x7fec4419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec43e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec44103ff0 0x7fec4419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:44120/0 (socket says 172.21.15.50:44120) 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec43e00640 1 -- 172.21.15.50:0/1565883225 learned_addr learned my addr 172.21.15.50:0/1565883225 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec3b400640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec441049a0 0x7fec4419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.826+0000 7fec48a00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec44109120 0x7fec4419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec43e00640 1 -- 172.21.15.50:0/1565883225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec44109120 msgr2=0x7fec4419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec43e00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec44109120 0x7fec4419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec43e00640 1 -- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec441049a0 msgr2=0x7fec4419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec43e00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec441049a0 0x7fec4419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec43e00640 1 -- 172.21.15.50:0/1565883225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4419f5d0 con 0x7fec44103ff0 2024-09-12T03:25:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec43e00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec44103ff0 0x7fec4419a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec34004770 tx=0x7fec34003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3403f070 con 0x7fec44103ff0 2024-09-12T03:25:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec34031ca0 con 0x7fec44103ff0 2024-09-12T03:25:52.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4419f860 con 0x7fec44103ff0 2024-09-12T03:25:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.827+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34035030 con 0x7fec44103ff0 2024-09-12T03:25:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.828+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441a71a0 con 0x7fec44103ff0 2024-09-12T03:25:52.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.828+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec44062940 con 0x7fec44103ff0 2024-09-12T03:25:52.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.831+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fec34042070 con 0x7fec44103ff0 2024-09-12T03:25:52.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.832+0000 7fec41600640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fec280a2c10 0x7fec280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.832+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fec340f0100 con 0x7fec44103ff0 2024-09-12T03:25:52.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.832+0000 7fec3b400640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fec280a2c10 0x7fec280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.832+0000 7fec41600640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] conn(0x7fec280adde0 0x7fec280b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.833+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 --> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec34038e90 con 0x7fec280adde0 2024-09-12T03:25:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.833+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fec340f04b0 con 0x7fec44103ff0 2024-09-12T03:25:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.833+0000 7fec48a00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] conn(0x7fec280adde0 0x7fec280b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.833+0000 7fec3b400640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fec280a2c10 0x7fec280a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec440fff90 tx=0x7fec2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.833+0000 7fec48a00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] conn(0x7fec280adde0 0x7fec280b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.834+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== osd.6 v2:172.21.15.50:6818/3669387354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec34038e90 con 0x7fec280adde0 2024-09-12T03:25:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:52 smithi082 ceph-mon[35046]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:25:52.856 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.858 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.861 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.860+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 --> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec44062dc0 con 0x7fec280adde0 2024-09-12T03:25:52.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.861+0000 7fec41600640 1 -- 172.21.15.50:0/1565883225 <== osd.6 v2:172.21.15.50:6818/3669387354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec44062dc0 con 0x7fec280adde0 2024-09-12T03:25:52.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.862+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] conn(0x7fec280adde0 msgr2=0x7fec280b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.862+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] conn(0x7fec280adde0 0x7fec280b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.863+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fec280a2c10 msgr2=0x7fec280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.863+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fec280a2c10 0x7fec280a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec440fff90 tx=0x7fec2c005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.863+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec44103ff0 msgr2=0x7fec4419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.863+0000 7fec4aa00640 1 --2- 172.21.15.50:0/1565883225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec44103ff0 0x7fec4419a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec34004770 tx=0x7fec34003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.863+0000 7fec48a00640 1 -- 172.21.15.50:0/1565883225 reap_dead start 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.864+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 shutdown_connections 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.864+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 >> 172.21.15.50:0/1565883225 conn(0x7fec440ffa90 msgr2=0x7fec44100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.864+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 shutdown_connections 2024-09-12T03:25:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:52.864+0000 7fec4aa00640 1 -- 172.21.15.50:0/1565883225 wait complete. 2024-09-12T03:25:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:53.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:52 smithi142 ceph-mon[35329]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:25:53.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.068+0000 7f7fbb200640 1 -- 172.21.15.50:0/3045887093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 msgr2=0x7f7fbc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:53.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.068+0000 7f7fbb200640 1 --2- 172.21.15.50:0/3045887093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 0x7f7fbc108830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009900 tx=0x7f7fa802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 -- 172.21.15.50:0/3045887093 shutdown_connections 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 --2- 172.21.15.50:0/3045887093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fbc108f60 0x7f7fbc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 --2- 172.21.15.50:0/3045887093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 0x7f7fbc108830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 --2- 172.21.15.50:0/3045887093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc103e30 0x7f7fbc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 -- 172.21.15.50:0/3045887093 >> 172.21.15.50:0/3045887093 conn(0x7f7fbc0ff8d0 msgr2=0x7f7fbc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 -- 172.21.15.50:0/3045887093 shutdown_connections 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.069+0000 7f7fbb200640 1 -- 172.21.15.50:0/3045887093 wait complete. 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 Processor -- start 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 -- start start 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fbc103e30 0x7f7fbc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 0x7f7fbc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc108f60 0x7f7fbc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:53.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113fc0 con 0x7f7fbc1047e0 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113e40 con 0x7f7fbc108f60 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fbb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc114140 con 0x7f7fbc103e30 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fb9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 0x7f7fbc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc108f60 0x7f7fbc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fb9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fbc103e30 0x7f7fbc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fb9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 0x7f7fbc19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48810/0 (socket says 172.21.15.50:48810) 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fb9400640 1 -- 172.21.15.50:0/2462204560 learned_addr learned my addr 172.21.15.50:0/2462204560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.070+0000 7f7fba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc108f60 0x7f7fbc19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:44146/0 (socket says 172.21.15.50:44146) 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fba800640 1 -- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fbc103e30 msgr2=0x7f7fbc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fba800640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fbc103e30 0x7f7fbc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fba800640 1 -- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 msgr2=0x7f7fbc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:53.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fba800640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fbc1047e0 0x7f7fbc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fba800640 1 -- 172.21.15.50:0/2462204560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc19f430 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fba800640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc108f60 0x7f7fbc19ed50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fd0 tx=0x7f7fb400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400ce20 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc19f720 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.071+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb4005020 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.072+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4010720 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.072+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1a7000 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.072+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fbc062980 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.075+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7fb4005540 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.076+0000 7f7fb2a00640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fa00a2c10 0x7f7fa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.076+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7fb40cfcf0 con 0x7f7fbc108f60 2024-09-12T03:25:53.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.076+0000 7f7fb9e00640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fa00a2c10 0x7f7fa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.076+0000 7f7fb2a00640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] conn(0x7f7fa00adde0 0x7f7fa00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.076+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 --> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fb4019e50 con 0x7f7fa00adde0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.076+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7fb40d0140 con 0x7f7fbc108f60 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.077+0000 7f7fb9e00640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fa00a2c10 0x7f7fa00a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009bd0 tx=0x7f7fac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.077+0000 7f7fb9400640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] conn(0x7f7fa00adde0 0x7f7fa00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.077+0000 7f7fb9400640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] conn(0x7f7fa00adde0 0x7f7fa00b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.078+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== osd.11 v2:172.21.15.142:6824/572198399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fb4019e50 con 0x7f7fa00adde0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.093+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 --> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fbc062e00 con 0x7f7fa00adde0 2024-09-12T03:25:53.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.093+0000 7f7fb2a00640 1 -- 172.21.15.50:0/2462204560 <== osd.11 v2:172.21.15.142:6824/572198399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7fbc062e00 con 0x7f7fa00adde0 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] conn(0x7f7fa00adde0 msgr2=0x7f7fa00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] conn(0x7f7fa00adde0 0x7f7fa00b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fa00a2c10 msgr2=0x7f7fa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fa00a2c10 0x7f7fa00a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009bd0 tx=0x7f7fac005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc108f60 msgr2=0x7f7fbc19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 --2- 172.21.15.50:0/2462204560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc108f60 0x7f7fbc19ed50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fd0 tx=0x7f7fb400ef10 comp rx=0 tx=0).stop 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fba800640 1 -- 172.21.15.50:0/2462204560 reap_dead start 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 shutdown_connections 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 >> 172.21.15.50:0/2462204560 conn(0x7f7fbc0ff8d0 msgr2=0x7f7fbc100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 shutdown_connections 2024-09-12T03:25:53.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.094+0000 7f7fbb200640 1 -- 172.21.15.50:0/2462204560 wait complete. 2024-09-12T03:25:54.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 -- 172.21.15.50:0/90627717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8108f60 msgr2=0x7f7ca8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 --2- 172.21.15.50:0/90627717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8108f60 0x7f7ca8110ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009be0 tx=0x7f7ca002a6d0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 -- 172.21.15.50:0/90627717 shutdown_connections 2024-09-12T03:25:54.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 --2- 172.21.15.50:0/90627717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8108f60 0x7f7ca8110ae0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 --2- 172.21.15.50:0/90627717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca81047e0 0x7f7ca8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 --2- 172.21.15.50:0/90627717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 0x7f7ca8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 -- 172.21.15.50:0/90627717 >> 172.21.15.50:0/90627717 conn(0x7f7ca80ff910 msgr2=0x7f7ca8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 -- 172.21.15.50:0/90627717 shutdown_connections 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.943+0000 7f7cade00640 1 -- 172.21.15.50:0/90627717 wait complete. 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.944+0000 7f7cade00640 1 Processor -- start 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.944+0000 7f7cade00640 1 -- start start 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.944+0000 7f7cade00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 0x7f7ca819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.944+0000 7f7cade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca81047e0 0x7f7ca819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.944+0000 7f7cade00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca8108f60 0x7f7ca819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7cade00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8113fc0 con 0x7f7ca8103e30 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7cade00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8113e40 con 0x7f7ca81047e0 2024-09-12T03:25:54.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7cade00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8114140 con 0x7f7ca8108f60 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 0x7f7ca819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 0x7f7ca819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48824/0 (socket says 172.21.15.50:48824) 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 -- 172.21.15.50:0/3704647192 learned_addr learned my addr 172.21.15.50:0/3704647192 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7cad400640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca8108f60 0x7f7ca819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7ca7e00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca81047e0 0x7f7ca819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 -- 172.21.15.50:0/3704647192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca8108f60 msgr2=0x7f7ca819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca8108f60 0x7f7ca819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 -- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca81047e0 msgr2=0x7f7ca819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca81047e0 0x7f7ca819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.945+0000 7f7caca00640 1 -- 172.21.15.50:0/3704647192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca819f3f0 con 0x7f7ca8103e30 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.946+0000 7f7caca00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 0x7f7ca819a440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.946+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00ccc0 con 0x7f7ca8103e30 2024-09-12T03:25:54.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.946+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c005500 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.946+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca819f6e0 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.946+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c010430 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.946+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca81a6fc0 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.947+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ca8062940 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.950+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7c9c0046c0 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.952+0000 7f7ca5600640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7c900a2c10 0x7f7c900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.952+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7c9c0cf860 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.952+0000 7f7ca7e00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7c900a2c10 0x7f7c900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.952+0000 7f7ca5600640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] conn(0x7f7c900adde0 0x7f7c900b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.952+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 --> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c900b0950 con 0x7f7c900adde0 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.952+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7c9c0d42a0 con 0x7f7ca8103e30 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.953+0000 7f7cad400640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] conn(0x7f7c900adde0 0x7f7c900b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.953+0000 7f7ca7e00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7c900a2c10 0x7f7c900a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80ffd30 tx=0x7f7c94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.953+0000 7f7cad400640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] conn(0x7f7c900adde0 0x7f7c900b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.954+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== osd.1 v2:172.21.15.82:6800/987070931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c900b0950 con 0x7f7c900adde0 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.969+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 --> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ca8062dc0 con 0x7f7c900adde0 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.970+0000 7f7ca5600640 1 -- 172.21.15.50:0/3704647192 <== osd.1 v2:172.21.15.82:6800/987070931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7ca8062dc0 con 0x7f7c900adde0 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.970+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] conn(0x7f7c900adde0 msgr2=0x7f7c900b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.970+0000 7f7cade00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] conn(0x7f7c900adde0 0x7f7c900b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7c900a2c10 msgr2=0x7f7c900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7c900a2c10 0x7f7c900a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80ffd30 tx=0x7f7c94005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 msgr2=0x7f7ca819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 --2- 172.21.15.50:0/3704647192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ca8103e30 0x7f7ca819a440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c00ee80 comp rx=0 tx=0).stop 2024-09-12T03:25:54.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cad400640 1 -- 172.21.15.50:0/3704647192 reap_dead start 2024-09-12T03:25:54.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 shutdown_connections 2024-09-12T03:25:54.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 >> 172.21.15.50:0/3704647192 conn(0x7f7ca80ff910 msgr2=0x7f7ca8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 shutdown_connections 2024-09-12T03:25:54.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:53.971+0000 7f7cade00640 1 -- 172.21.15.50:0/3704647192 wait complete. 2024-09-12T03:25:54.855 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:54 smithi050 ceph-mon[30378]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:25:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:54 smithi142 ceph-mon[35329]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 -- 172.21.15.50:0/183246619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 msgr2=0x7f34d01124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 --2- 172.21.15.50:0/183246619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01124f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009900 tx=0x7f34b802a410 comp rx=0 tx=0).stop 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 -- 172.21.15.50:0/183246619 shutdown_connections 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 --2- 172.21.15.50:0/183246619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0112a30 0x7f34d0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 --2- 172.21.15.50:0/183246619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01124f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 --2- 172.21.15.50:0/183246619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34d0076720 0x7f34d0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 -- 172.21.15.50:0/183246619 >> 172.21.15.50:0/183246619 conn(0x7f34d00ffa90 msgr2=0x7f34d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 -- 172.21.15.50:0/183246619 shutdown_connections 2024-09-12T03:25:55.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.079+0000 7f34d5600640 1 -- 172.21.15.50:0/183246619 wait complete. 2024-09-12T03:25:55.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 Processor -- start 2024-09-12T03:25:55.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 -- start start 2024-09-12T03:25:55.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34d0076720 0x7f34d01a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0112a30 0x7f34d019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34d01185c0 con 0x7f34d0077040 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0118440 con 0x7f34d0112a30 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.080+0000 7f34d5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0118740 con 0x7f34d0076720 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48838/0 (socket says 172.21.15.50:48838) 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 -- 172.21.15.50:0/3844420081 learned_addr learned my addr 172.21.15.50:0/3844420081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34cea00640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34d0076720 0x7f34d01a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34cf400640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0112a30 0x7f34d019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 -- 172.21.15.50:0/3844420081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34d0076720 msgr2=0x7f34d01a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34d0076720 0x7f34d01a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 -- 172.21.15.50:0/3844420081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0112a30 msgr2=0x7f34d019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0112a30 0x7f34d019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34c7e00640 1 -- 172.21.15.50:0/3844420081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d019f000 con 0x7f34d0077040 2024-09-12T03:25:55.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.081+0000 7f34cf400640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0112a30 0x7f34d019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:55.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.082+0000 7f34c7e00640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01a4c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006010 tx=0x7f34b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:55.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.082+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b803f030 con 0x7f34d0077040 2024-09-12T03:25:55.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.082+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b8003ce0 con 0x7f34d0077040 2024-09-12T03:25:55.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.082+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d019f290 con 0x7f34d0077040 2024-09-12T03:25:55.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.082+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8035070 con 0x7f34d0077040 2024-09-12T03:25:55.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.082+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d01ab5e0 con 0x7f34d0077040 2024-09-12T03:25:55.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.083+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34d0114a30 con 0x7f34d0077040 2024-09-12T03:25:55.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.085+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f34b8042030 con 0x7f34d0077040 2024-09-12T03:25:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.086+0000 7f34c7400640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f34b40a2c10 0x7f34b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f34b80f0140 con 0x7f34d0077040 2024-09-12T03:25:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34cea00640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f34b40a2c10 0x7f34b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34c7400640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] conn(0x7f34b40adde0 0x7f34b40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 --> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34b8040e70 con 0x7f34b40adde0 2024-09-12T03:25:55.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f34b80f04f0 con 0x7f34d0077040 2024-09-12T03:25:55.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34cf400640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] conn(0x7f34b40adde0 0x7f34b40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.087+0000 7f34cea00640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f34b40a2c10 0x7f34b40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f34d019fa10 tx=0x7f34c000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:55.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.088+0000 7f34cf400640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] conn(0x7f34b40adde0 0x7f34b40b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.089+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== osd.3 v2:172.21.15.50:6810/2709499694 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34b8040e70 con 0x7f34b40adde0 2024-09-12T03:25:55.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.103+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 --> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34d0077e60 con 0x7f34b40adde0 2024-09-12T03:25:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:54 smithi082 ceph-mon[35046]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:25:55.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.103+0000 7f34c7400640 1 -- 172.21.15.50:0/3844420081 <== osd.3 v2:172.21.15.50:6810/2709499694 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34d0077e60 con 0x7f34b40adde0 2024-09-12T03:25:55.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] conn(0x7f34b40adde0 msgr2=0x7f34b40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] conn(0x7f34b40adde0 0x7f34b40b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f34b40a2c10 msgr2=0x7f34b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f34b40a2c10 0x7f34b40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f34d019fa10 tx=0x7f34c000b390 comp rx=0 tx=0).stop 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 msgr2=0x7f34d01a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 --2- 172.21.15.50:0/3844420081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d0077040 0x7f34d01a4c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006010 tx=0x7f34b80042f0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34cf400640 1 -- 172.21.15.50:0/3844420081 reap_dead start 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 shutdown_connections 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.104+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 >> 172.21.15.50:0/3844420081 conn(0x7f34d00ffa90 msgr2=0x7f34d0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.105+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 shutdown_connections 2024-09-12T03:25:55.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.105+0000 7f34d5600640 1 -- 172.21.15.50:0/3844420081 wait complete. 2024-09-12T03:25:55.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 -- 172.21.15.50:0/2179029518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c076720 msgr2=0x7f0a1c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 --2- 172.21.15.50:0/2179029518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c076720 0x7f0a1c076b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009860 tx=0x7f0a0c02a340 comp rx=0 tx=0).stop 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 -- 172.21.15.50:0/2179029518 shutdown_connections 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 --2- 172.21.15.50:0/2179029518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c112a30 0x7f0a1c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 --2- 172.21.15.50:0/2179029518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a1c077040 0x7f0a1c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 --2- 172.21.15.50:0/2179029518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c076720 0x7f0a1c076b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.318+0000 7f0a21c00640 1 -- 172.21.15.50:0/2179029518 >> 172.21.15.50:0/2179029518 conn(0x7f0a1c0ffa90 msgr2=0x7f0a1c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.319+0000 7f0a21c00640 1 -- 172.21.15.50:0/2179029518 shutdown_connections 2024-09-12T03:25:55.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.319+0000 7f0a21c00640 1 -- 172.21.15.50:0/2179029518 wait complete. 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 Processor -- start 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 -- start start 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c076720 0x7f0a1c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a1c077040 0x7f0a1c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c112a30 0x7f0a1c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1185a0 con 0x7f0a1c076720 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c118420 con 0x7f0a1c112a30 2024-09-12T03:25:55.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a21c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c118720 con 0x7f0a1c077040 2024-09-12T03:25:55.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a1aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c076720 0x7f0a1c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a1aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c076720 0x7f0a1c1a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48866/0 (socket says 172.21.15.50:48866) 2024-09-12T03:25:55.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a1aa00640 1 -- 172.21.15.50:0/950054169 learned_addr learned my addr 172.21.15.50:0/950054169 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:55.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.320+0000 7f0a1b400640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c112a30 0x7f0a1c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1a000640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a1c077040 0x7f0a1c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1aa00640 1 -- 172.21.15.50:0/950054169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a1c077040 msgr2=0x7f0a1c1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1aa00640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a1c077040 0x7f0a1c1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1aa00640 1 -- 172.21.15.50:0/950054169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c112a30 msgr2=0x7f0a1c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1aa00640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c112a30 0x7f0a1c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1aa00640 1 -- 172.21.15.50:0/950054169 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c19efc0 con 0x7f0a1c076720 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1b400640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c112a30 0x7f0a1c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a1aa00640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c076720 0x7f0a1c1a46d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c035040 tx=0x7f0a0c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.321+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c037040 con 0x7f0a1c076720 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.322+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c040070 con 0x7f0a1c076720 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.322+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c19f250 con 0x7f0a1c076720 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.322+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003a50 con 0x7f0a1c076720 2024-09-12T03:25:55.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.322+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1ab5a0 con 0x7f0a1c076720 2024-09-12T03:25:55.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.323+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a1c062980 con 0x7f0a1c076720 2024-09-12T03:25:55.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.325+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0a0c003bf0 con 0x7f0a1c076720 2024-09-12T03:25:55.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.326+0000 7f0a13400640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0a000a2c10 0x7f0a000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.326+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0a0c0eff70 con 0x7f0a1c076720 2024-09-12T03:25:55.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.326+0000 7f0a1a000640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0a000a2c10 0x7f0a000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.327+0000 7f0a13400640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] conn(0x7f0a000adcb0 0x7f0a000b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.327+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 --> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a0c034e90 con 0x7f0a000adcb0 2024-09-12T03:25:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.327+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c0f03c0 con 0x7f0a1c076720 2024-09-12T03:25:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.327+0000 7f0a1b400640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] conn(0x7f0a000adcb0 0x7f0a000b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.327+0000 7f0a1a000640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0a000a2c10 0x7f0a000a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c079370 tx=0x7f0a0800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:55.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.328+0000 7f0a1b400640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] conn(0x7f0a000adcb0 0x7f0a000b0150 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:55.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.328+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== osd.0 v2:172.21.15.50:6802/894870617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a0c034e90 con 0x7f0a000adcb0 2024-09-12T03:25:55.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.343+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 --> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a1c062e00 con 0x7f0a000adcb0 2024-09-12T03:25:55.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a13400640 1 -- 172.21.15.50:0/950054169 <== osd.0 v2:172.21.15.50:6802/894870617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a1c062e00 con 0x7f0a000adcb0 2024-09-12T03:25:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] conn(0x7f0a000adcb0 msgr2=0x7f0a000b0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a21c00640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] conn(0x7f0a000adcb0 0x7f0a000b0150 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:55.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0a000a2c10 msgr2=0x7f0a000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a21c00640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0a000a2c10 0x7f0a000a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c079370 tx=0x7f0a0800b390 comp rx=0 tx=0).stop 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c076720 msgr2=0x7f0a1c1a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a21c00640 1 --2- 172.21.15.50:0/950054169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a1c076720 0x7f0a1c1a46d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c035040 tx=0x7f0a0c033d90 comp rx=0 tx=0).stop 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.344+0000 7f0a1b400640 1 -- 172.21.15.50:0/950054169 reap_dead start 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.345+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 shutdown_connections 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.345+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 >> 172.21.15.50:0/950054169 conn(0x7f0a1c0ffa90 msgr2=0x7f0a1c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.345+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 shutdown_connections 2024-09-12T03:25:55.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:55.345+0000 7f0a21c00640 1 -- 172.21.15.50:0/950054169 wait complete. 2024-09-12T03:25:56.377 INFO:teuthology.orchestra.run.smithi050.stdout:107374182473 2024-09-12T03:25:56.377 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T03:25:56.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:56 smithi050 ceph-mon[30378]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:25:57.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:56 smithi142 ceph-mon[35329]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:25:57.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:56 smithi082 ceph-mon[35046]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:25:57.140 INFO:teuthology.orchestra.run.smithi050.stdout:55834574931 2024-09-12T03:25:57.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 -- 172.21.15.50:0/179614946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef8102b90 msgr2=0x7fdef8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 --2- 172.21.15.50:0/179614946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef8102b90 0x7fdef8102f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009900 tx=0x7fdeec02a480 comp rx=0 tx=0).stop 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 -- 172.21.15.50:0/179614946 shutdown_connections 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 --2- 172.21.15.50:0/179614946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef810bfc0 0x7fdef810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 --2- 172.21.15.50:0/179614946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef81034b0 0x7fdef810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 --2- 172.21.15.50:0/179614946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef8102b90 0x7fdef8102f70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 -- 172.21.15.50:0/179614946 >> 172.21.15.50:0/179614946 conn(0x7fdef80fc610 msgr2=0x7fdef80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 -- 172.21.15.50:0/179614946 shutdown_connections 2024-09-12T03:25:57.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.916+0000 7fdefe400640 1 -- 172.21.15.50:0/179614946 wait complete. 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.917+0000 7fdefe400640 1 Processor -- start 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.917+0000 7fdefe400640 1 -- start start 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefe400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef8102b90 0x7fdef819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefe400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef81034b0 0x7fdef819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefe400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef810bfc0 0x7fdef819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefe400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8111dc0 con 0x7fdef8102b90 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefe400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8111c40 con 0x7fdef810bfc0 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefe400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8111f40 con 0x7fdef81034b0 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefda00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef810bfc0 0x7fdef819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefda00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef810bfc0 0x7fdef819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:44196/0 (socket says 172.21.15.50:44196) 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef8102b90 0x7fdef819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdefda00640 1 -- 172.21.15.50:0/2835761405 learned_addr learned my addr 172.21.15.50:0/2835761405 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:57.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.918+0000 7fdef7e00640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef81034b0 0x7fdef819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefda00640 1 -- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef81034b0 msgr2=0x7fdef819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefda00640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef81034b0 0x7fdef819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefda00640 1 -- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef8102b90 msgr2=0x7fdef819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefda00640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdef8102b90 0x7fdef819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefda00640 1 -- 172.21.15.50:0/2835761405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef819f460 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefda00640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef810bfc0 0x7fdef819ed80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdef000efd0 tx=0x7fdef000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0003b70 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef819f750 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef0003d10 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00109f0 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.919+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef81a7030 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.920+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdef81022a0 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.922+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdef0010b90 con 0x7fdef810bfc0 2024-09-12T03:25:57.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.924+0000 7fdef5600640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdee00a2c90 0x7fdee00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.924+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdef00cfc90 con 0x7fdef810bfc0 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.924+0000 7fdefd000640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdee00a2c90 0x7fdee00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.924+0000 7fdef5600640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] conn(0x7fdee00ade60 0x7fdee00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.924+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 --> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdef0009820 con 0x7fdee00ade60 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.924+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fdef00d0040 con 0x7fdef810bfc0 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.925+0000 7fdef7e00640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] conn(0x7fdee00ade60 0x7fdee00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.925+0000 7fdefd000640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdee00a2c90 0x7fdee00a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009230 tx=0x7fdeec0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.925+0000 7fdef7e00640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] conn(0x7fdee00ade60 0x7fdee00b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.926+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== osd.8 v2:172.21.15.142:6816/1190406046 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdef0009820 con 0x7fdee00ade60 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.940+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 --> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdef806b070 con 0x7fdee00ade60 2024-09-12T03:25:57.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdef5600640 1 -- 172.21.15.50:0/2835761405 <== osd.8 v2:172.21.15.142:6816/1190406046 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdef806b070 con 0x7fdee00ade60 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] conn(0x7fdee00ade60 msgr2=0x7fdee00b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefe400640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] conn(0x7fdee00ade60 0x7fdee00b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdee00a2c90 msgr2=0x7fdee00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefe400640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdee00a2c90 0x7fdee00a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009230 tx=0x7fdeec0058d0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef810bfc0 msgr2=0x7fdef819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefe400640 1 --2- 172.21.15.50:0/2835761405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef810bfc0 0x7fdef819ed80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdef000efd0 tx=0x7fdef000c6a0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.941+0000 7fdefda00640 1 -- 172.21.15.50:0/2835761405 reap_dead start 2024-09-12T03:25:57.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.942+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 shutdown_connections 2024-09-12T03:25:57.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.942+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 >> 172.21.15.50:0/2835761405 conn(0x7fdef80fc610 msgr2=0x7fdef810d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.942+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 shutdown_connections 2024-09-12T03:25:57.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:56.942+0000 7fdefe400640 1 -- 172.21.15.50:0/2835761405 wait complete. 2024-09-12T03:25:58.033 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:58.035 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:58.035 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:58.038 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:58.038 INFO:teuthology.orchestra.run.smithi050.stdout:34359738457 2024-09-12T03:25:58.039 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T03:25:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:58.406 INFO:teuthology.orchestra.run.smithi050.stdout:240518168624 2024-09-12T03:25:58.407 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T03:25:58.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:25:58 smithi050 ceph-mon[30378]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:25:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:25:58 smithi142 ceph-mon[35329]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:25:59.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:25:58 smithi082 ceph-mon[35046]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:25:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 -- 172.21.15.50:0/3871187641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4106f10 msgr2=0x7fa6b410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 --2- 172.21.15.50:0/3871187641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4106f10 0x7fa6b410ea90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009ca0 tx=0x7fa6ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 -- 172.21.15.50:0/3871187641 shutdown_connections 2024-09-12T03:25:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 --2- 172.21.15.50:0/3871187641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4106f10 0x7fa6b410ea90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 --2- 172.21.15.50:0/3871187641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b4102790 0x7fa6b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 --2- 172.21.15.50:0/3871187641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b4101de0 0x7fa6b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 -- 172.21.15.50:0/3871187641 >> 172.21.15.50:0/3871187641 conn(0x7fa6b40fd8c0 msgr2=0x7fa6b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 -- 172.21.15.50:0/3871187641 shutdown_connections 2024-09-12T03:25:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.349+0000 7fa6b9200640 1 -- 172.21.15.50:0/3871187641 wait complete. 2024-09-12T03:25:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 Processor -- start 2024-09-12T03:25:59.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 -- start start 2024-09-12T03:25:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4101de0 0x7fa6b41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b4102790 0x7fa6b4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b4106f10 0x7fa6b419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4111f70 con 0x7fa6b4101de0 2024-09-12T03:25:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4111df0 con 0x7fa6b4106f10 2024-09-12T03:25:59.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.351+0000 7fa6b9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41120f0 con 0x7fa6b4102790 2024-09-12T03:25:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6b1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4101de0 0x7fa6b41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6b1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4101de0 0x7fa6b41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45744/0 (socket says 172.21.15.50:45744) 2024-09-12T03:25:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6b1e00640 1 -- 172.21.15.50:0/1453692746 learned_addr learned my addr 172.21.15.50:0/1453692746 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6abe00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b4102790 0x7fa6b4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6b2800640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b4106f10 0x7fa6b419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6abe00640 1 -- 172.21.15.50:0/1453692746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b4106f10 msgr2=0x7fa6b419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6abe00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b4106f10 0x7fa6b419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6abe00640 1 -- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4101de0 msgr2=0x7fa6b41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6abe00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b4101de0 0x7fa6b41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.352+0000 7fa6abe00640 1 -- 172.21.15.50:0/1453692746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b419d3a0 con 0x7fa6b4102790 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.353+0000 7fa6abe00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b4102790 0x7fa6b4198930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9f0 tx=0x7fa69c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.353+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00ccc0 con 0x7fa6b4102790 2024-09-12T03:25:59.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.353+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b419d690 con 0x7fa6b4102790 2024-09-12T03:25:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.353+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69c005020 con 0x7fa6b4102790 2024-09-12T03:25:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.353+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0049e0 con 0x7fa6b4102790 2024-09-12T03:25:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.353+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a4f70 con 0x7fa6b4102790 2024-09-12T03:25:59.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.354+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6b4062900 con 0x7fa6b4102790 2024-09-12T03:25:59.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.356+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa69c005560 con 0x7fa6b4102790 2024-09-12T03:25:59.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.358+0000 7fa6aaa00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6980a2c10 0x7fa6980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.358+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa69c0cf5e0 con 0x7fa6b4102790 2024-09-12T03:25:59.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.358+0000 7fa6b1e00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6980a2c10 0x7fa6980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.358+0000 7fa6aaa00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] conn(0x7fa6980adde0 0x7fa6980b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.358+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 --> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6980b0950 con 0x7fa6980adde0 2024-09-12T03:25:59.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.358+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa69c0cf990 con 0x7fa6b4102790 2024-09-12T03:25:59.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.359+0000 7fa6b2800640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] conn(0x7fa6980adde0 0x7fa6980b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.359+0000 7fa6b1e00640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6980a2c10 0x7fa6980a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40095f0 tx=0x7fa6a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.360+0000 7fa6b2800640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] conn(0x7fa6980adde0 0x7fa6980b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.361+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== osd.9 v2:172.21.15.50:6826/1350383902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6980b0950 con 0x7fa6980adde0 2024-09-12T03:25:59.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.376+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 --> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6b4062d80 con 0x7fa6980adde0 2024-09-12T03:25:59.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.377+0000 7fa6aaa00640 1 -- 172.21.15.50:0/1453692746 <== osd.9 v2:172.21.15.50:6826/1350383902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa6b4062d80 con 0x7fa6980adde0 2024-09-12T03:25:59.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.377+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] conn(0x7fa6980adde0 msgr2=0x7fa6980b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.377+0000 7fa6b9200640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] conn(0x7fa6980adde0 0x7fa6980b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6980a2c10 msgr2=0x7fa6980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6980a2c10 0x7fa6980a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40095f0 tx=0x7fa6a4005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b4102790 msgr2=0x7fa6b4198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 --2- 172.21.15.50:0/1453692746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b4102790 0x7fa6b4198930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9f0 tx=0x7fa69c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b2800640 1 -- 172.21.15.50:0/1453692746 reap_dead start 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 shutdown_connections 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 >> 172.21.15.50:0/1453692746 conn(0x7fa6b40fd8c0 msgr2=0x7fa6b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 shutdown_connections 2024-09-12T03:25:59.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.378+0000 7fa6b9200640 1 -- 172.21.15.50:0/1453692746 wait complete. 2024-09-12T03:25:59.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 -- 172.21.15.50:0/1810288982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81079c0 msgr2=0x7f32c8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 --2- 172.21.15.50:0/1810288982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81079c0 0x7f32c8107da0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009930 tx=0x7f32bc02a420 comp rx=0 tx=0).stop 2024-09-12T03:25:59.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 -- 172.21.15.50:0/1810288982 shutdown_connections 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 --2- 172.21.15.50:0/1810288982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c8102360 0x7f32c810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 --2- 172.21.15.50:0/1810288982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c81019c0 0x7f32c8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 --2- 172.21.15.50:0/1810288982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81079c0 0x7f32c8107da0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 -- 172.21.15.50:0/1810288982 >> 172.21.15.50:0/1810288982 conn(0x7f32c80fd740 msgr2=0x7f32c80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 -- 172.21.15.50:0/1810288982 shutdown_connections 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.245+0000 7f32ce000640 1 -- 172.21.15.50:0/1810288982 wait complete. 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.246+0000 7f32ce000640 1 Processor -- start 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.246+0000 7f32ce000640 1 -- start start 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ce000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81019c0 0x7f32c819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ce000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c8102360 0x7f32c819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ce000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c81079c0 0x7f32c819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ce000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111e10 con 0x7f32c81019c0 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ce000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111c90 con 0x7f32c8102360 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ce000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111f90 con 0x7f32c81079c0 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32ccc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81019c0 0x7f32c819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c8102360 0x7f32c819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c8102360 0x7f32c819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34492/0 (socket says 172.21.15.50:34492) 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32c7e00640 1 -- 172.21.15.50:0/4148410559 learned_addr learned my addr 172.21.15.50:0/4148410559 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32cd600640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c81079c0 0x7f32c819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32c7e00640 1 -- 172.21.15.50:0/4148410559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c81079c0 msgr2=0x7f32c819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.247+0000 7f32c7e00640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c81079c0 0x7f32c819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c7e00640 1 -- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81019c0 msgr2=0x7f32c819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c7e00640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c81019c0 0x7f32c819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c7e00640 1 -- 172.21.15.50:0/4148410559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c819f460 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c7e00640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c8102360 0x7f32c819a9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e9f0 tx=0x7f32b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b400cd80 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b4010910 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b40188f0 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c819f750 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.248+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c81a7030 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.249+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32c80fe7f0 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.252+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32b4010ac0 con 0x7f32c8102360 2024-09-12T03:25:59.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.253+0000 7f32c5600640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f32b00a2c10 0x7f32b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.254+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32b40cffa0 con 0x7f32c8102360 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.254+0000 7f32ccc00640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f32b00a2c10 0x7f32b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.254+0000 7f32c5600640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] conn(0x7f32b00adde0 0x7f32b00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.254+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 --> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32b00b0950 con 0x7f32b00adde0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.254+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f32b40d0350 con 0x7f32c8102360 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.254+0000 7f32cd600640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] conn(0x7f32b00adde0 0x7f32b00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.255+0000 7f32ccc00640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f32b00a2c10 0x7f32b00a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006010 tx=0x7f32bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.255+0000 7f32cd600640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] conn(0x7f32b00adde0 0x7f32b00b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.256+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== osd.10 v2:172.21.15.82:6824/1817816537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32b00b0950 con 0x7f32b00adde0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.270+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 --> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32c8103100 con 0x7f32b00adde0 2024-09-12T03:25:59.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.271+0000 7f32c5600640 1 -- 172.21.15.50:0/4148410559 <== osd.10 v2:172.21.15.82:6824/1817816537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32c8103100 con 0x7f32b00adde0 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.271+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] conn(0x7f32b00adde0 msgr2=0x7f32b00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.271+0000 7f32ce000640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] conn(0x7f32b00adde0 0x7f32b00b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.271+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f32b00a2c10 msgr2=0x7f32b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f32b00a2c10 0x7f32b00a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006010 tx=0x7f32bc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c8102360 msgr2=0x7f32c819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 --2- 172.21.15.50:0/4148410559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32c8102360 0x7f32c819a9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e9f0 tx=0x7f32b400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32cd600640 1 -- 172.21.15.50:0/4148410559 reap_dead start 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 shutdown_connections 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 >> 172.21.15.50:0/4148410559 conn(0x7f32c80fd740 msgr2=0x7f32c80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 shutdown_connections 2024-09-12T03:25:59.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.272+0000 7f32ce000640 1 -- 172.21.15.50:0/4148410559 wait complete. 2024-09-12T03:26:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 -- 172.21.15.50:0/230250804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9001049a0 msgr2=0x7fc9001089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 --2- 172.21.15.50:0/230250804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9001049a0 0x7fc9001089f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009900 tx=0x7fc8ec02a410 comp rx=0 tx=0).stop 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 -- 172.21.15.50:0/230250804 shutdown_connections 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 --2- 172.21.15.50:0/230250804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc900109120 0x7fc900110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 --2- 172.21.15.50:0/230250804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9001049a0 0x7fc9001089f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 --2- 172.21.15.50:0/230250804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900103ff0 0x7fc9001043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 -- 172.21.15.50:0/230250804 >> 172.21.15.50:0/230250804 conn(0x7fc9000ffa90 msgr2=0x7fc900101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.212+0000 7fc906e00640 1 -- 172.21.15.50:0/230250804 shutdown_connections 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.213+0000 7fc906e00640 1 -- 172.21.15.50:0/230250804 wait complete. 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.213+0000 7fc906e00640 1 Processor -- start 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 -- start start 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc900103ff0 0x7fc90019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9001049a0 0x7fc90019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc900109120 0x7fc90019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc900114140 con 0x7fc900103ff0 2024-09-12T03:26:00.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc900113fc0 con 0x7fc9001049a0 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc906e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001142c0 con 0x7fc900109120 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc900103ff0 0x7fc90019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc900103ff0 0x7fc90019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45784/0 (socket says 172.21.15.50:45784) 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc8ffe00640 1 -- 172.21.15.50:0/925296603 learned_addr learned my addr 172.21.15.50:0/925296603 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc8ff400640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9001049a0 0x7fc90019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.214+0000 7fc904e00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc900109120 0x7fc90019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ffe00640 1 -- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc900109120 msgr2=0x7fc90019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ffe00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc900109120 0x7fc90019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ffe00640 1 -- 172.21.15.50:0/925296603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9001049a0 msgr2=0x7fc90019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ffe00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9001049a0 0x7fc90019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ffe00640 1 -- 172.21.15.50:0/925296603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90019f5d0 con 0x7fc900103ff0 2024-09-12T03:26:00.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ff400640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9001049a0 0x7fc90019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:00.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8ffe00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc900103ff0 0x7fc90019a620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000e9f0 tx=0x7fc8f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:00.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000cd80 con 0x7fc900103ff0 2024-09-12T03:26:00.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f0010910 con 0x7fc900103ff0 2024-09-12T03:26:00.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90019f8c0 con 0x7fc900103ff0 2024-09-12T03:26:00.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.215+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0018990 con 0x7fc900103ff0 2024-09-12T03:26:00.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.216+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a71a0 con 0x7fc900103ff0 2024-09-12T03:26:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.217+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc900062980 con 0x7fc900103ff0 2024-09-12T03:26:00.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.218+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc8f0010ac0 con 0x7fc900103ff0 2024-09-12T03:26:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.220+0000 7fc8fcc00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc8e40a2c90 0x7fc8e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.220+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc8f00cf800 con 0x7fc900103ff0 2024-09-12T03:26:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.220+0000 7fc8fcc00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] conn(0x7fc8e40ade60 0x7fc8e40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.220+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 --> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8f0012ec0 con 0x7fc8e40ade60 2024-09-12T03:26:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.220+0000 7fc8ff400640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc8e40a2c90 0x7fc8e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.220+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc8f00d6050 con 0x7fc900103ff0 2024-09-12T03:26:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.221+0000 7fc904e00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] conn(0x7fc8e40ade60 0x7fc8e40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.221+0000 7fc8ff400640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc8e40a2c90 0x7fc8e40a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009230 tx=0x7fc8ec0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:00.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.221+0000 7fc904e00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] conn(0x7fc8e40ade60 0x7fc8e40b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:00.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.222+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== osd.2 v2:172.21.15.142:6800/1535997813 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8f0012ec0 con 0x7fc8e40ade60 2024-09-12T03:26:00.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.238+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 --> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc900062d80 con 0x7fc8e40ade60 2024-09-12T03:26:00.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc8fcc00640 1 -- 172.21.15.50:0/925296603 <== osd.2 v2:172.21.15.142:6800/1535997813 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc900062d80 con 0x7fc8e40ade60 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] conn(0x7fc8e40ade60 msgr2=0x7fc8e40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] conn(0x7fc8e40ade60 0x7fc8e40b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc8e40a2c90 msgr2=0x7fc8e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc8e40a2c90 0x7fc8e40a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009230 tx=0x7fc8ec0058d0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc900103ff0 msgr2=0x7fc90019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 --2- 172.21.15.50:0/925296603 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc900103ff0 0x7fc90019a620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000e9f0 tx=0x7fc8f000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc904e00640 1 -- 172.21.15.50:0/925296603 reap_dead start 2024-09-12T03:26:00.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 shutdown_connections 2024-09-12T03:26:00.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.239+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 >> 172.21.15.50:0/925296603 conn(0x7fc9000ffa90 msgr2=0x7fc900100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:00.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.240+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 shutdown_connections 2024-09-12T03:26:00.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:00.240+0000 7fc906e00640 1 -- 172.21.15.50:0/925296603 wait complete. 2024-09-12T03:26:00.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 -- 172.21.15.50:0/3755702882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 msgr2=0x7f1d341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/3755702882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d341021c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009900 tx=0x7f1d2402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 -- 172.21.15.50:0/3755702882 shutdown_connections 2024-09-12T03:26:00.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/3755702882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d34106f10 0x7f1d3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/3755702882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d34102790 0x7f1d341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/3755702882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d341021c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 -- 172.21.15.50:0/3755702882 >> 172.21.15.50:0/3755702882 conn(0x7f1d340fd8c0 msgr2=0x7f1d340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 -- 172.21.15.50:0/3755702882 shutdown_connections 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.721+0000 7f1d3ac00640 1 -- 172.21.15.50:0/3755702882 wait complete. 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.722+0000 7f1d3ac00640 1 Processor -- start 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.722+0000 7f1d3ac00640 1 -- start start 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d3ac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d3ac00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d3ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d3ac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34111fc0 con 0x7f1d34101de0 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d3ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34111e40 con 0x7f1d34106f10 2024-09-12T03:26:00.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d3ac00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d34112140 con 0x7f1d34102790 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d33e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d33400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d33e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45766/0 (socket says 172.21.15.50:45766) 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d33e00640 1 -- 172.21.15.50:0/4199638120 learned_addr learned my addr 172.21.15.50:0/4199638120 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d38c00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.723+0000 7f1d33e00640 1 -- 172.21.15.50:0/4199638120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d34102790 msgr2=0x7f1d34074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d33e00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d34102790 0x7f1d34074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d33e00640 1 -- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d34106f10 msgr2=0x7f1d34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d33e00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d34106f10 0x7f1d34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d33e00640 1 -- 172.21.15.50:0/4199638120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d341a6900 con 0x7f1d34101de0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d33e00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d24004770 tx=0x7f1d24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2403f070 con 0x7f1d34101de0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d24031ca0 con 0x7f1d34101de0 2024-09-12T03:26:00.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24035030 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d341a6b90 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.724+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d341a70c0 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.725+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d340628c0 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.727+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d24042070 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.728+0000 7f1d30c00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d180a2c90 0x7f1d180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.728+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1d240efff0 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.729+0000 7f1d33400640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d180a2c90 0x7f1d180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.729+0000 7f1d30c00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] conn(0x7f1d180ade60 0x7f1d180b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.729+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 --> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d24032e50 con 0x7f1d180ade60 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.729+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f1d240f0400 con 0x7f1d34101de0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.729+0000 7f1d38c00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] conn(0x7f1d180ade60 0x7f1d180b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:00.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.729+0000 7f1d33400640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d180a2c90 0x7f1d180a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d34075f80 tx=0x7f1d2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.730+0000 7f1d38c00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] conn(0x7f1d180ade60 0x7f1d180b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.731+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== osd.7 v2:172.21.15.82:6816/704851047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d24032e50 con 0x7f1d180ade60 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.745+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 --> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d34062cc0 con 0x7f1d180ade60 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.745+0000 7f1d30c00640 1 -- 172.21.15.50:0/4199638120 <== osd.7 v2:172.21.15.82:6816/704851047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d34062cc0 con 0x7f1d180ade60 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] conn(0x7f1d180ade60 msgr2=0x7f1d180b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] conn(0x7f1d180ade60 0x7f1d180b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d180a2c90 msgr2=0x7f1d180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d180a2c90 0x7f1d180a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d34075f80 tx=0x7f1d2000b410 comp rx=0 tx=0).stop 2024-09-12T03:26:00.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 msgr2=0x7f1d340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 --2- 172.21.15.50:0/4199638120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d34101de0 0x7f1d340744f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d24004770 tx=0x7f1d24003c10 comp rx=0 tx=0).stop 2024-09-12T03:26:00.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d38c00640 1 -- 172.21.15.50:0/4199638120 reap_dead start 2024-09-12T03:26:00.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 shutdown_connections 2024-09-12T03:26:00.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.746+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 >> 172.21.15.50:0/4199638120 conn(0x7f1d340fd8c0 msgr2=0x7f1d340febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:00.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.747+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 shutdown_connections 2024-09-12T03:26:00.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:25:59.747+0000 7f1d3ac00640 1 -- 172.21.15.50:0/4199638120 wait complete. 2024-09-12T03:26:00.869 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:00 smithi050 ceph-mon[30378]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:26:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:00 smithi142 ceph-mon[35329]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:26:01.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:00 smithi082 ceph-mon[35046]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:26:01.356 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:01.373 INFO:teuthology.orchestra.run.smithi050.stdout:210453397557 2024-09-12T03:26:01.373 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 -- 172.21.15.50:0/1489763039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898077040 msgr2=0x7f68981124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 --2- 172.21.15.50:0/1489763039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898077040 0x7f68981124f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6880009900 tx=0x7f688002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 -- 172.21.15.50:0/1489763039 shutdown_connections 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 --2- 172.21.15.50:0/1489763039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898112a30 0x7f6898114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 --2- 172.21.15.50:0/1489763039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898077040 0x7f68981124f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 --2- 172.21.15.50:0/1489763039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898076720 0x7f6898076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 -- 172.21.15.50:0/1489763039 >> 172.21.15.50:0/1489763039 conn(0x7f68980ffa90 msgr2=0x7f6898101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 -- 172.21.15.50:0/1489763039 shutdown_connections 2024-09-12T03:26:01.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.711+0000 7f689ee00640 1 -- 172.21.15.50:0/1489763039 wait complete. 2024-09-12T03:26:01.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 Processor -- start 2024-09-12T03:26:01.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 -- start start 2024-09-12T03:26:01.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898076720 0x7f68981a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898077040 0x7f68981a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898112a30 0x7f689819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68981185b0 con 0x7f6898112a30 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6898118430 con 0x7f6898077040 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.712+0000 7f689ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6898118730 con 0x7f6898076720 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f689ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898112a30 0x7f689819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898076720 0x7f68981a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898076720 0x7f68981a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60728/0 (socket says 172.21.15.50:60728) 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 -- 172.21.15.50:0/692008415 learned_addr learned my addr 172.21.15.50:0/692008415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f689ce00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898112a30 0x7f689819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45800/0 (socket says 172.21.15.50:45800) 2024-09-12T03:26:01.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f688fe00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898077040 0x7f68981a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:01.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 -- 172.21.15.50:0/692008415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898077040 msgr2=0x7f68981a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:01.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898077040 0x7f68981a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 -- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898112a30 msgr2=0x7f689819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898112a30 0x7f689819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f6897e00640 1 -- 172.21.15.50:0/692008415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689819eff0 con 0x7f6898076720 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.713+0000 7f689ce00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6898112a30 0x7f689819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.714+0000 7f6897e00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898076720 0x7f68981a4700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f688800d950 tx=0x7f688800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.714+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888017030 con 0x7f6898076720 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.714+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6888004020 con 0x7f6898076720 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.714+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689819f2e0 con 0x7f6898076720 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.714+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880047a0 con 0x7f6898076720 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.714+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981ab5d0 con 0x7f6898076720 2024-09-12T03:26:01.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.715+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6898062980 con 0x7f6898076720 2024-09-12T03:26:01.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.717+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f68880041d0 con 0x7f6898076720 2024-09-12T03:26:01.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.718+0000 7f6895600640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f687c0a2c10 0x7f687c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:01.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.718+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f68880cf590 con 0x7f6898076720 2024-09-12T03:26:01.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.718+0000 7f688fe00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f687c0a2c10 0x7f687c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:01.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.719+0000 7f6895600640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] conn(0x7f687c0adde0 0x7f687c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:01.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.719+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 --> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f687c0b0950 con 0x7f687c0adde0 2024-09-12T03:26:01.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.719+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f68880cf940 con 0x7f6898076720 2024-09-12T03:26:01.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.719+0000 7f689ce00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] conn(0x7f687c0adde0 0x7f687c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:01.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.719+0000 7f688fe00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f687c0a2c10 0x7f687c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6880009230 tx=0x7f68800058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:01.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.720+0000 7f689ce00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] conn(0x7f687c0adde0 0x7f687c0b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:01.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.721+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== osd.5 v2:172.21.15.142:6808/361252032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f687c0b0950 con 0x7f687c0adde0 2024-09-12T03:26:01.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.738+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 --> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6898062e00 con 0x7f687c0adde0 2024-09-12T03:26:01.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.739+0000 7f6895600640 1 -- 172.21.15.50:0/692008415 <== osd.5 v2:172.21.15.142:6808/361252032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6898062e00 con 0x7f687c0adde0 2024-09-12T03:26:01.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.739+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] conn(0x7f687c0adde0 msgr2=0x7f687c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:01.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.739+0000 7f689ee00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] conn(0x7f687c0adde0 0x7f687c0b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f687c0a2c10 msgr2=0x7f687c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:01.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f687c0a2c10 0x7f687c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6880009230 tx=0x7f68800058d0 comp rx=0 tx=0).stop 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898076720 msgr2=0x7f68981a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 --2- 172.21.15.50:0/692008415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898076720 0x7f68981a4700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f688800d950 tx=0x7f688800de10 comp rx=0 tx=0).stop 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ce00640 1 -- 172.21.15.50:0/692008415 reap_dead start 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 shutdown_connections 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 >> 172.21.15.50:0/692008415 conn(0x7f68980ffa90 msgr2=0x7f6898101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 shutdown_connections 2024-09-12T03:26:01.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:01.740+0000 7f689ee00640 1 -- 172.21.15.50:0/692008415 wait complete. 2024-09-12T03:26:02.897 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:02 smithi050 ceph-mon[30378]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:26:02.906 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:02 smithi082 ceph-mon[35046]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:26:02.936 INFO:teuthology.orchestra.run.smithi050.stdout:77309411407 2024-09-12T03:26:02.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T03:26:03.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:02 smithi142 ceph-mon[35329]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:26:03.605 INFO:teuthology.orchestra.run.smithi050.stdout:330712481822 2024-09-12T03:26:03.606 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T03:26:04.101 INFO:teuthology.orchestra.run.smithi050.stdout:188978561080 2024-09-12T03:26:04.101 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T03:26:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:04 smithi050 ceph-mon[30378]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:04.744 INFO:teuthology.orchestra.run.smithi050.stdout:158913790016 2024-09-12T03:26:04.745 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T03:26:04.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 -- 172.21.15.50:0/3730836980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa34410bfc0 msgr2=0x7fa34410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 --2- 172.21.15.50:0/3730836980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa34410bfc0 0x7fa34410e460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009be0 tx=0x7fa33c02a750 comp rx=0 tx=0).stop 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 -- 172.21.15.50:0/3730836980 shutdown_connections 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 --2- 172.21.15.50:0/3730836980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa34410bfc0 0x7fa34410e460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 --2- 172.21.15.50:0/3730836980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3441034b0 0x7fa34410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 --2- 172.21.15.50:0/3730836980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344102b90 0x7fa344102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 -- 172.21.15.50:0/3730836980 >> 172.21.15.50:0/3730836980 conn(0x7fa3440fc610 msgr2=0x7fa3440fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 -- 172.21.15.50:0/3730836980 shutdown_connections 2024-09-12T03:26:04.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.922+0000 7fa349200640 1 -- 172.21.15.50:0/3730836980 wait complete. 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.923+0000 7fa349200640 1 Processor -- start 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 -- start start 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344102b90 0x7fa3441039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3441034b0 0x7fa344103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa34410bfc0 0x7fa344104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa344111dc0 con 0x7fa34410bfc0 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa344111c40 con 0x7fa3441034b0 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa349200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa344111f40 con 0x7fa344102b90 2024-09-12T03:26:04.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa342a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3441034b0 0x7fa344103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:04.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa343e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa34410bfc0 0x7fa344104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:04.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa342a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3441034b0 0x7fa344103f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34602/0 (socket says 172.21.15.50:34602) 2024-09-12T03:26:04.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa342a00640 1 -- 172.21.15.50:0/3905916484 learned_addr learned my addr 172.21.15.50:0/3905916484 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:04.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.924+0000 7fa343400640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344102b90 0x7fa3441039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:04.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa343e00640 1 -- 172.21.15.50:0/3905916484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344102b90 msgr2=0x7fa3441039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa343e00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344102b90 0x7fa3441039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa343e00640 1 -- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3441034b0 msgr2=0x7fa344103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa343e00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3441034b0 0x7fa344103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa343e00640 1 -- 172.21.15.50:0/3905916484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3441aaa90 con 0x7fa34410bfc0 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa343e00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa34410bfc0 0x7fa344104460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa33c034040 tx=0x7fa33c0305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.925+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c041020 con 0x7fa34410bfc0 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.926+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3441aad20 con 0x7fa34410bfc0 2024-09-12T03:26:04.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.926+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441ab250 con 0x7fa34410bfc0 2024-09-12T03:26:04.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.926+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33c003e50 con 0x7fa34410bfc0 2024-09-12T03:26:04.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.926+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c036040 con 0x7fa34410bfc0 2024-09-12T03:26:04.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.927+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa324001030 con 0x7fa34410bfc0 2024-09-12T03:26:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.930+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa33c044020 con 0x7fa34410bfc0 2024-09-12T03:26:04.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.931+0000 7fa33be00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa3280a2c90 0x7fa3280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:04.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.932+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa33c0f0400 con 0x7fa34410bfc0 2024-09-12T03:26:04.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.932+0000 7fa343400640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa3280a2c90 0x7fa3280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.932+0000 7fa33be00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] conn(0x7fa3280ade60 0x7fa3280b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.932+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 --> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3280b09d0 con 0x7fa3280ade60 2024-09-12T03:26:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.932+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa33c0f07b0 con 0x7fa34410bfc0 2024-09-12T03:26:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.932+0000 7fa342a00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] conn(0x7fa3280ade60 0x7fa3280b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.933+0000 7fa343400640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa3280a2c90 0x7fa3280a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa33400bbe0 tx=0x7fa33400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:04.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.933+0000 7fa342a00640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] conn(0x7fa3280ade60 0x7fa3280b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:04.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.934+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== osd.4 v2:172.21.15.82:6808/2469208064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3280b09d0 con 0x7fa3280ade60 2024-09-12T03:26:04.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.948+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 --> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa324002dd0 con 0x7fa3280ade60 2024-09-12T03:26:04.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.949+0000 7fa33be00640 1 -- 172.21.15.50:0/3905916484 <== osd.4 v2:172.21.15.82:6808/2469208064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa324002dd0 con 0x7fa3280ade60 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.949+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] conn(0x7fa3280ade60 msgr2=0x7fa3280b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.949+0000 7fa349200640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] conn(0x7fa3280ade60 0x7fa3280b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa3280a2c90 msgr2=0x7fa3280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa3280a2c90 0x7fa3280a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa33400bbe0 tx=0x7fa33400b390 comp rx=0 tx=0).stop 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa34410bfc0 msgr2=0x7fa344104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 --2- 172.21.15.50:0/3905916484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa34410bfc0 0x7fa344104460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa33c034040 tx=0x7fa33c0305b0 comp rx=0 tx=0).stop 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa343e00640 1 -- 172.21.15.50:0/3905916484 reap_dead start 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 shutdown_connections 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 >> 172.21.15.50:0/3905916484 conn(0x7fa3440fc610 msgr2=0x7fa3440fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 shutdown_connections 2024-09-12T03:26:04.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.950+0000 7fa349200640 1 -- 172.21.15.50:0/3905916484 wait complete. 2024-09-12T03:26:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:04 smithi142 ceph-mon[35329]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:04 smithi082 ceph-mon[35046]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:300647710756 2024-09-12T03:26:05.391 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T03:26:05.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:05 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3439583698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:26:06.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:05 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3439583698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:26:06.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:05 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3439583698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:26:06.251 INFO:teuthology.orchestra.run.smithi050.stdout:274877906985 2024-09-12T03:26:06.252 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T03:26:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 -- 172.21.15.50:0/4262711904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c106f10 msgr2=0x7fcd6c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 --2- 172.21.15.50:0/4262711904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c106f10 0x7fcd6c10ea90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009ca0 tx=0x7fcd6402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 -- 172.21.15.50:0/4262711904 shutdown_connections 2024-09-12T03:26:06.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 --2- 172.21.15.50:0/4262711904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c106f10 0x7fcd6c10ea90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 --2- 172.21.15.50:0/4262711904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c102790 0x7fcd6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 --2- 172.21.15.50:0/4262711904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c101de0 0x7fcd6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 -- 172.21.15.50:0/4262711904 >> 172.21.15.50:0/4262711904 conn(0x7fcd6c0fd8c0 msgr2=0x7fcd6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 -- 172.21.15.50:0/4262711904 shutdown_connections 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.507+0000 7fcd71c00640 1 -- 172.21.15.50:0/4262711904 wait complete. 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 Processor -- start 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 -- start start 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 0x7fcd6c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c102790 0x7fcd6c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c106f10 0x7fcd6c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c111fc0 con 0x7fcd6c101de0 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c111e40 con 0x7fcd6c102790 2024-09-12T03:26:06.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd71c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c112140 con 0x7fcd6c106f10 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.508+0000 7fcd6aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 0x7fcd6c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 0x7fcd6c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45824/0 (socket says 172.21.15.50:45824) 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 -- 172.21.15.50:0/3439583698 learned_addr learned my addr 172.21.15.50:0/3439583698 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd63e00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c102790 0x7fcd6c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6b400640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c106f10 0x7fcd6c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 -- 172.21.15.50:0/3439583698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c106f10 msgr2=0x7fcd6c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c106f10 0x7fcd6c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 -- 172.21.15.50:0/3439583698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c102790 msgr2=0x7fcd6c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c102790 0x7fcd6c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 -- 172.21.15.50:0/3439583698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6c19f5b0 con 0x7fcd6c101de0 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6b400640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c106f10 0x7fcd6c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.509+0000 7fcd6aa00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 0x7fcd6c19a600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00e9f0 tx=0x7fcd5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:06.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.510+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00cd80 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.510+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c010910 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.510+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c018990 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.510+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6c19f8a0 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.510+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6c1a7180 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.511+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6c1021c0 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.513+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcd5c010ac0 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.514+0000 7fcd63400640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcd500a2c10 0x7fcd500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.514+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcd5c0cf970 con 0x7fcd6c101de0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.514+0000 7fcd63e00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcd500a2c10 0x7fcd500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.515+0000 7fcd63e00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcd500a2c10 0x7fcd500a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c108890 tx=0x7fcd54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:06.325 INFO:teuthology.orchestra.run.smithi050.stdout:107374182474 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.518+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd5c09c3b0 con 0x7fcd6c101de0 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.711+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcd6c1a7350 con 0x7fcd6c101de0 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.712+0000 7fcd63400640 1 -- 172.21.15.50:0/3439583698 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd5c09bcb0 con 0x7fcd6c101de0 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.714+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcd500a2c10 msgr2=0x7fcd500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.714+0000 7fcd71c00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcd500a2c10 0x7fcd500a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c108890 tx=0x7fcd54005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.714+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 msgr2=0x7fcd6c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.714+0000 7fcd71c00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 0x7fcd6c19a600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00e9f0 tx=0x7fcd5c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 shutdown_connections 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcd500a2c10 0x7fcd500a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd6c106f10 0x7fcd6c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c102790 0x7fcd6c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 --2- 172.21.15.50:0/3439583698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd6c101de0 0x7fcd6c19a600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 >> 172.21.15.50:0/3439583698 conn(0x7fcd6c0fd8c0 msgr2=0x7fcd6c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 shutdown_connections 2024-09-12T03:26:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:04.715+0000 7fcd71c00640 1 -- 172.21.15.50:0/3439583698 wait complete. 2024-09-12T03:26:06.625 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:06 smithi050 ceph-mon[30378]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:06 smithi142 ceph-mon[35329]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:07.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:06 smithi082 ceph-mon[35046]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:07.197 INFO:teuthology.orchestra.run.smithi050.stdout:137438953542 2024-09-12T03:26:07.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T03:26:07.680 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182473 got 107374182474 for osd.3 2024-09-12T03:26:07.681 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:08.077 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.077 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.077 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.078 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.078 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.082 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.111 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 -- 172.21.15.50:0/897750281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 msgr2=0x7fefa0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 --2- 172.21.15.50:0/897750281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 0x7fefa0074a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef90009900 tx=0x7fef9002a480 comp rx=0 tx=0).stop 2024-09-12T03:26:08.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 -- 172.21.15.50:0/897750281 shutdown_connections 2024-09-12T03:26:08.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 --2- 172.21.15.50:0/897750281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa010e210 0x7fefa0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 --2- 172.21.15.50:0/897750281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa0074f40 0x7fefa010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 --2- 172.21.15.50:0/897750281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 0x7fefa0074a00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 -- 172.21.15.50:0/897750281 >> 172.21.15.50:0/897750281 conn(0x7fefa00fd710 msgr2=0x7fefa00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:08.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 -- 172.21.15.50:0/897750281 shutdown_connections 2024-09-12T03:26:08.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.360+0000 7fefa5600640 1 -- 172.21.15.50:0/897750281 wait complete. 2024-09-12T03:26:08.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.361+0000 7fefa5600640 1 Processor -- start 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 -- start start 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 0x7fefa019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 0x7fefa019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa010e210 0x7fefa019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0114010 con 0x7fefa0074620 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0113e90 con 0x7fefa010e210 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0114190 con 0x7fefa0074f40 2024-09-12T03:26:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fef9f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 0x7fefa019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.362+0000 7fefa4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa010e210 0x7fefa019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 0x7fefa019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:60800/0 (socket says 172.21.15.50:60800) 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 0x7fefa019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 -- 172.21.15.50:0/1749073845 learned_addr learned my addr 172.21.15.50:0/1749073845 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 -- 172.21.15.50:0/1749073845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa010e210 msgr2=0x7fefa019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa010e210 0x7fefa019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 -- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 msgr2=0x7fefa019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 0x7fefa019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 -- 172.21.15.50:0/1749073845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa019f450 con 0x7fefa0074f40 2024-09-12T03:26:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.363+0000 7fef9f400640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 0x7fefa019a9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9f0 tx=0x7fef8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.364+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00cd80 con 0x7fefa0074f40 2024-09-12T03:26:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.364+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef8c010910 con 0x7fefa0074f40 2024-09-12T03:26:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.364+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa019f740 con 0x7fefa0074f40 2024-09-12T03:26:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.364+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c018990 con 0x7fefa0074f40 2024-09-12T03:26:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.364+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01a7020 con 0x7fefa0074f40 2024-09-12T03:26:08.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.366+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa0074a00 con 0x7fefa0074f40 2024-09-12T03:26:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.367+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fef8c010ac0 con 0x7fefa0074f40 2024-09-12T03:26:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.370+0000 7fef9cc00640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fef840a2c90 0x7fef840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.370+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fef8c0cf800 con 0x7fefa0074f40 2024-09-12T03:26:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.370+0000 7fef9fe00640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fef840a2c90 0x7fef840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:08.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.371+0000 7fef9fe00640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fef840a2c90 0x7fef840a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef90000c00 tx=0x7fef90005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.373+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef8c09bbf0 con 0x7fefa0074f40 2024-09-12T03:26:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.567+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fefa01093f0 con 0x7fefa0074f40 2024-09-12T03:26:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.568+0000 7fef9cc00640 1 -- 172.21.15.50:0/1749073845 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fef8c09fef0 con 0x7fefa0074f40 2024-09-12T03:26:08.568 INFO:teuthology.orchestra.run.smithi050.stdout:55834574934 2024-09-12T03:26:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.570+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fef840a2c90 msgr2=0x7fef840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.570+0000 7fefa5600640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fef840a2c90 0x7fef840a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef90000c00 tx=0x7fef90005b50 comp rx=0 tx=0).stop 2024-09-12T03:26:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.570+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 msgr2=0x7fefa019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.570+0000 7fefa5600640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 0x7fefa019a9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9f0 tx=0x7fef8c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 shutdown_connections 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fef840a2c90 0x7fef840a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa010e210 0x7fefa019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa0074f40 0x7fefa019a9e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 --2- 172.21.15.50:0/1749073845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefa0074620 0x7fefa019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 >> 172.21.15.50:0/1749073845 conn(0x7fefa00fd710 msgr2=0x7fefa00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 shutdown_connections 2024-09-12T03:26:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:08.571+0000 7fefa5600640 1 -- 172.21.15.50:0/1749073845 wait complete. 2024-09-12T03:26:08.859 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:08 smithi050 ceph-mon[30378]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:08.860 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:08 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1749073845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:26:09.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:08 smithi142 ceph-mon[35329]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:09.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:08 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1749073845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 -- 172.21.15.50:0/3780018250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 msgr2=0x7f79f00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3780018250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f00fffc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009900 tx=0x7f79dc02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 -- 172.21.15.50:0/3780018250 shutdown_connections 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3780018250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3780018250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f00fffc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3780018250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f0106b80 0x7f79f0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 -- 172.21.15.50:0/3780018250 >> 172.21.15.50:0/3780018250 conn(0x7f79f00fba90 msgr2=0x7f79f00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 -- 172.21.15.50:0/3780018250 shutdown_connections 2024-09-12T03:26:09.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.060+0000 7f79f7a00640 1 -- 172.21.15.50:0/3780018250 wait complete. 2024-09-12T03:26:09.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.061+0000 7f79f7a00640 1 Processor -- start 2024-09-12T03:26:09.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 -- start start 2024-09-12T03:26:09.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f0106b80 0x7f79f019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111fb0 con 0x7f79f00ffb60 2024-09-12T03:26:09.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111e30 con 0x7f79f0100500 2024-09-12T03:26:09.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f7a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0112130 con 0x7f79f0106b80 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79f5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.062+0000 7f79efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79f5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53450/0 (socket says 172.21.15.50:53450) 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:45328/0 (socket says 172.21.15.50:45328) 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79f5000640 1 -- 172.21.15.50:0/3590451733 learned_addr learned my addr 172.21.15.50:0/3590451733 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79f5a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f0106b80 0x7f79f019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 -- 172.21.15.50:0/3590451733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f0106b80 msgr2=0x7f79f019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f0106b80 0x7f79f019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 -- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 msgr2=0x7f79f019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 -- 172.21.15.50:0/3590451733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f019f630 con 0x7f79f0100500 2024-09-12T03:26:09.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.063+0000 7f79efe00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f019abc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79dc02a910 tx=0x7f79dc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:09.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.064+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc03f070 con 0x7f79f0100500 2024-09-12T03:26:09.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.064+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79dc031c20 con 0x7f79f0100500 2024-09-12T03:26:09.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.064+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f019f8c0 con 0x7f79f0100500 2024-09-12T03:26:09.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.064+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc035030 con 0x7f79f0100500 2024-09-12T03:26:09.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.064+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a7200 con 0x7f79f0100500 2024-09-12T03:26:09.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.066+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f0101080 con 0x7f79f0100500 2024-09-12T03:26:09.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.069+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f79dc042070 con 0x7f79f0100500 2024-09-12T03:26:09.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.070+0000 7f79ed600640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f79d80a2c10 0x7f79d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.070+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f79dc0efcf0 con 0x7f79f0100500 2024-09-12T03:26:09.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.070+0000 7f79f5000640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f79d80a2c10 0x7f79d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.071+0000 7f79f5000640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f79d80a2c10 0x7f79d80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005fd0 tx=0x7f79e400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:09.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.074+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79dc0bc0e0 con 0x7f79f0100500 2024-09-12T03:26:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:08 smithi082 ceph-mon[35046]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:08 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1749073845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:26:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.265+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f79f01a7400 con 0x7f79f0100500 2024-09-12T03:26:09.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.266+0000 7f79ed600640 1 -- 172.21.15.50:0/3590451733 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f79dc0c03e0 con 0x7f79f0100500 2024-09-12T03:26:09.266 INFO:teuthology.orchestra.run.smithi050.stdout:274877906986 2024-09-12T03:26:09.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f79d80a2c10 msgr2=0x7f79d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f79d80a2c10 0x7f79d80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005fd0 tx=0x7f79e400b4a0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 msgr2=0x7f79f019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f019abc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79dc02a910 tx=0x7f79dc003b90 comp rx=0 tx=0).stop 2024-09-12T03:26:09.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 shutdown_connections 2024-09-12T03:26:09.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f79d80a2c10 0x7f79d80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f0106b80 0x7f79f019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f0100500 0x7f79f019abc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 --2- 172.21.15.50:0/3590451733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f00ffb60 0x7f79f019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 >> 172.21.15.50:0/3590451733 conn(0x7f79f00fba90 msgr2=0x7f79f010d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:09.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 shutdown_connections 2024-09-12T03:26:09.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:09.268+0000 7f79f7a00640 1 -- 172.21.15.50:0/3590451733 wait complete. 2024-09-12T03:26:09.735 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:09 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3590451733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:26:10.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:09 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3590451733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:26:10.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:09 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3590451733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:26:10.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 -- 172.21.15.50:0/3574101449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 msgr2=0x7f6f680748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:10.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 --2- 172.21.15.50:0/3574101449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f680748f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009860 tx=0x7f6f5802a3d0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 -- 172.21.15.50:0/3574101449 shutdown_connections 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 --2- 172.21.15.50:0/3574101449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f6810e280 0x7f6f68110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 --2- 172.21.15.50:0/3574101449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f68074e30 0x7f6f6810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 --2- 172.21.15.50:0/3574101449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f680748f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.131+0000 7f6f67200640 1 -- 172.21.15.50:0/3574101449 >> 172.21.15.50:0/3574101449 conn(0x7f6f680fd740 msgr2=0x7f6f680ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.132+0000 7f6f67200640 1 -- 172.21.15.50:0/3574101449 shutdown_connections 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.132+0000 7f6f67200640 1 -- 172.21.15.50:0/3574101449 wait complete. 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.133+0000 7f6f67200640 1 Processor -- start 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.133+0000 7f6f67200640 1 -- start start 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f67200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f6819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f67200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f68074e30 0x7f6f6819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f67200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f6810e280 0x7f6f6819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f67200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68114000 con 0x7f6f68074e30 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f67200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68113e80 con 0x7f6f68074510 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f67200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68114180 con 0x7f6f6810e280 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f65e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f6819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f65e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f6819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:45368/0 (socket says 172.21.15.50:45368) 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f68074e30 0x7f6f6819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f65e00640 1 -- 172.21.15.50:0/4095162913 learned_addr learned my addr 172.21.15.50:0/4095162913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.134+0000 7f6f66800640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f6810e280 0x7f6f6819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f65e00640 1 -- 172.21.15.50:0/4095162913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f6810e280 msgr2=0x7f6f6819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f65e00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f6810e280 0x7f6f6819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f65e00640 1 -- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f68074e30 msgr2=0x7f6f6819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f65e00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f68074e30 0x7f6f6819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f65e00640 1 -- 172.21.15.50:0/4095162913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6819f450 con 0x7f6f68074510 2024-09-12T03:26:10.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f5fe00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f68074e30 0x7f6f6819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f65e00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f6819a4a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009980 tx=0x7f6f580309a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58040070 con 0x7f6f68074510 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f58005620 con 0x7f6f68074510 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580049b0 con 0x7f6f68074510 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.135+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6819f6e0 con 0x7f6f68074510 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.136+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681a7020 con 0x7f6f68074510 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.137+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f680748f0 con 0x7f6f68074510 2024-09-12T03:26:10.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.139+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6f58043070 con 0x7f6f68074510 2024-09-12T03:26:10.401 INFO:teuthology.orchestra.run.smithi050.stdout:137438953543 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.141+0000 7f6f5ea00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f6f4c0a2c90 0x7f6f4c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.142+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6f580f05f0 con 0x7f6f68074510 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.142+0000 7f6f5fe00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f6f4c0a2c90 0x7f6f4c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.142+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f580bc9e0 con 0x7f6f68074510 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.143+0000 7f6f5fe00640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f6f4c0a2c90 0x7f6f4c0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f500095f0 tx=0x7f6f50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.346+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6f68109400 con 0x7f6f68074510 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.347+0000 7f6f5ea00640 1 -- 172.21.15.50:0/4095162913 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6f580c0ce0 con 0x7f6f68074510 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f6f4c0a2c90 msgr2=0x7f6f4c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f6f4c0a2c90 0x7f6f4c0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f500095f0 tx=0x7f6f50005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 msgr2=0x7f6f6819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:10.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f6819a4a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009980 tx=0x7f6f580309a0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 shutdown_connections 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f6f4c0a2c90 0x7f6f4c0a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f6810e280 0x7f6f6819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f68074e30 0x7f6f6819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 --2- 172.21.15.50:0/4095162913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f68074510 0x7f6f6819a4a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 >> 172.21.15.50:0/4095162913 conn(0x7f6f680fd740 msgr2=0x7f6f680fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 shutdown_connections 2024-09-12T03:26:10.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.349+0000 7f6f67200640 1 -- 172.21.15.50:0/4095162913 wait complete. 2024-09-12T03:26:10.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:10 smithi050 ceph-mon[30378]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:26:10.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:10 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3682673202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:26:10.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:10 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4095162913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:26:11.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:10 smithi142 ceph-mon[35329]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:26:11.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:10 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3682673202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:26:11.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:10 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4095162913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:26:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:10 smithi082 ceph-mon[35046]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:26:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:10 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3682673202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:26:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:10 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4095162913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 -- 172.21.15.50:0/3252781824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40106f20 msgr2=0x7f7b4010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 --2- 172.21.15.50:0/3252781824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40106f20 0x7f7b4010eaa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009ca0 tx=0x7f7b3802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 -- 172.21.15.50:0/3252781824 shutdown_connections 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 --2- 172.21.15.50:0/3252781824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40106f20 0x7f7b4010eaa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 --2- 172.21.15.50:0/3252781824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b401027a0 0x7f7b401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 --2- 172.21.15.50:0/3252781824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b40101df0 0x7f7b401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 -- 172.21.15.50:0/3252781824 >> 172.21.15.50:0/3252781824 conn(0x7f7b400fd8b0 msgr2=0x7f7b400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 -- 172.21.15.50:0/3252781824 shutdown_connections 2024-09-12T03:26:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.534+0000 7f7b48400640 1 -- 172.21.15.50:0/3252781824 wait complete. 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 Processor -- start 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 -- start start 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 0x7f7b4019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b401027a0 0x7f7b4019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b40106f20 0x7f7b4019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40111fd0 con 0x7f7b40101df0 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40111e50 con 0x7f7b401027a0 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.535+0000 7f7b48400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40112150 con 0x7f7b40106f20 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 0x7f7b4019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 0x7f7b4019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53544/0 (socket says 172.21.15.50:53544) 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 -- 172.21.15.50:0/233700534 learned_addr learned my addr 172.21.15.50:0/233700534 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45000640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b401027a0 0x7f7b4019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b46400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b40106f20 0x7f7b4019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 -- 172.21.15.50:0/233700534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b40106f20 msgr2=0x7f7b4019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b40106f20 0x7f7b4019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 -- 172.21.15.50:0/233700534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b401027a0 msgr2=0x7f7b4019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b401027a0 0x7f7b4019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 -- 172.21.15.50:0/233700534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4019f590 con 0x7f7b40101df0 2024-09-12T03:26:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.536+0000 7f7b45a00640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 0x7f7b4019a5e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400e9f0 tx=0x7f7b3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:11.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.537+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3400cd80 con 0x7f7b40101df0 2024-09-12T03:26:11.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.537+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b34010910 con 0x7f7b40101df0 2024-09-12T03:26:11.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.537+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34018990 con 0x7f7b40101df0 2024-09-12T03:26:11.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.537+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4019f880 con 0x7f7b40101df0 2024-09-12T03:26:11.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.537+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b401a7160 con 0x7f7b40101df0 2024-09-12T03:26:11.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.538+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b401021d0 con 0x7f7b40101df0 2024-09-12T03:26:11.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.540+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b34010ac0 con 0x7f7b40101df0 2024-09-12T03:26:11.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.541+0000 7f7b3e000640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7b280a2c90 0x7f7b280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.541+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b340cf8b0 con 0x7f7b40101df0 2024-09-12T03:26:11.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.541+0000 7f7b45000640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7b280a2c90 0x7f7b280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.542+0000 7f7b45000640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7b280a2c90 0x7f7b280a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7b400ff610 tx=0x7f7b2c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:11.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.545+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b3409c2f0 con 0x7f7b40101df0 2024-09-12T03:26:11.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.739+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7b401a73c0 con 0x7f7b40101df0 2024-09-12T03:26:11.739 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:11 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2902529073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:26:11.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.740+0000 7f7b3e000640 1 -- 172.21.15.50:0/233700534 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7b3409bbf0 con 0x7f7b40101df0 2024-09-12T03:26:11.740 INFO:teuthology.orchestra.run.smithi050.stdout:240518168627 2024-09-12T03:26:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7b280a2c90 msgr2=0x7f7b280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7b280a2c90 0x7f7b280a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7b400ff610 tx=0x7f7b2c00b410 comp rx=0 tx=0).stop 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 msgr2=0x7f7b4019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 0x7f7b4019a5e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400e9f0 tx=0x7f7b3400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 shutdown_connections 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7b280a2c90 0x7f7b280a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b40106f20 0x7f7b4019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b401027a0 0x7f7b4019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 --2- 172.21.15.50:0/233700534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b40101df0 0x7f7b4019a5e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.742+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 >> 172.21.15.50:0/233700534 conn(0x7f7b400fd8b0 msgr2=0x7f7b400fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.743+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 shutdown_connections 2024-09-12T03:26:11.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.743+0000 7f7b48400640 1 -- 172.21.15.50:0/233700534 wait complete. 2024-09-12T03:26:11.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 -- 172.21.15.50:0/2443886044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8076720 msgr2=0x7fb7a8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 --2- 172.21.15.50:0/2443886044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8076720 0x7fb7a8076b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009900 tx=0x7fb79c02a400 comp rx=0 tx=0).stop 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 -- 172.21.15.50:0/2443886044 shutdown_connections 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 --2- 172.21.15.50:0/2443886044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a8112a30 0x7fb7a8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 --2- 172.21.15.50:0/2443886044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7a8077040 0x7fb7a81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 --2- 172.21.15.50:0/2443886044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8076720 0x7fb7a8076b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 -- 172.21.15.50:0/2443886044 >> 172.21.15.50:0/2443886044 conn(0x7fb7a80ffa90 msgr2=0x7fb7a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 -- 172.21.15.50:0/2443886044 shutdown_connections 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.827+0000 7fb7af800640 1 -- 172.21.15.50:0/2443886044 wait complete. 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.828+0000 7fb7af800640 1 Processor -- start 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.828+0000 7fb7af800640 1 -- start start 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7af800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a8076720 0x7fb7a81a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7af800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7a8077040 0x7fb7a81a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7af800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 0x7fb7a819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7af800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8118610 con 0x7fb7a8112a30 2024-09-12T03:26:11.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7af800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8118490 con 0x7fb7a8077040 2024-09-12T03:26:11.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7af800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8118790 con 0x7fb7a8076720 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 0x7fb7a819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ace00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a8076720 0x7fb7a81a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 0x7fb7a819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53524/0 (socket says 172.21.15.50:53524) 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 -- 172.21.15.50:0/2902529073 learned_addr learned my addr 172.21.15.50:0/2902529073 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7a7e00640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7a8077040 0x7fb7a81a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 -- 172.21.15.50:0/2902529073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a8076720 msgr2=0x7fb7a81a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a8076720 0x7fb7a81a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 -- 172.21.15.50:0/2902529073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7a8077040 msgr2=0x7fb7a81a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7a8077040 0x7fb7a81a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.829+0000 7fb7ad800640 1 -- 172.21.15.50:0/2902529073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a819f010 con 0x7fb7a8112a30 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.830+0000 7fb7ad800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 0x7fb7a819e8e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000eb10 tx=0x7fb7a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.830+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0004b20 con 0x7fb7a8112a30 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.830+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0004cc0 con 0x7fb7a8112a30 2024-09-12T03:26:11.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.830+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0010790 con 0x7fb7a8112a30 2024-09-12T03:26:11.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.830+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a819f300 con 0x7fb7a8112a30 2024-09-12T03:26:11.879 INFO:teuthology.orchestra.run.smithi050.stdout:300647710758 2024-09-12T03:26:11.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.830+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a81ab5f0 con 0x7fb7a8112a30 2024-09-12T03:26:11.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.831+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a8076b00 con 0x7fb7a8112a30 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.833+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb7a0005020 con 0x7fb7a8112a30 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.835+0000 7fb7a5600640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb7900a2c90 0x7fb7900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.835+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb7a00cfbe0 con 0x7fb7a8112a30 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.835+0000 7fb7ace00640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb7900a2c90 0x7fb7900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.836+0000 7fb7ace00640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb7900a2c90 0x7fb7900a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb79c006010 tx=0x7fb79c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.839+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7a009c620 con 0x7fb7a8112a30 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.027+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb7a802d7c0 con 0x7fb7a8112a30 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.027+0000 7fb7a5600640 1 -- 172.21.15.50:0/2902529073 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb7a009bf20 con 0x7fb7a8112a30 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb7900a2c90 msgr2=0x7fb7900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb7900a2c90 0x7fb7900a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb79c006010 tx=0x7fb79c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 msgr2=0x7fb7a819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 0x7fb7a819e8e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000eb10 tx=0x7fb7a000c2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 shutdown_connections 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fb7900a2c90 0x7fb7900a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7a8112a30 0x7fb7a819e8e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7a8077040 0x7fb7a81a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 --2- 172.21.15.50:0/2902529073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a8076720 0x7fb7a81a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 >> 172.21.15.50:0/2902529073 conn(0x7fb7a80ffa90 msgr2=0x7fb7a8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 shutdown_connections 2024-09-12T03:26:11.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:11.029+0000 7fb7af800640 1 -- 172.21.15.50:0/2902529073 wait complete. 2024-09-12T03:26:12.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:11 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2902529073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:26:12.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:11 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2902529073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:26:12.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.039+0000 7f300ea00640 1 -- 172.21.15.50:0/2934759886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 msgr2=0x7f30081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.039+0000 7f300ea00640 1 --2- 172.21.15.50:0/2934759886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f30081043d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009900 tx=0x7f2ff802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 -- 172.21.15.50:0/2934759886 shutdown_connections 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 --2- 172.21.15.50:0/2934759886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3008109120 0x7f3008110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 --2- 172.21.15.50:0/2934759886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30081049a0 0x7f30081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 --2- 172.21.15.50:0/2934759886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f30081043d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 -- 172.21.15.50:0/2934759886 >> 172.21.15.50:0/2934759886 conn(0x7f30080ffa90 msgr2=0x7f3008101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 -- 172.21.15.50:0/2934759886 shutdown_connections 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.040+0000 7f300ea00640 1 -- 172.21.15.50:0/2934759886 wait complete. 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 Processor -- start 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 -- start start 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f300819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30081049a0 0x7f300819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3008109120 0x7f300819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3008114150 con 0x7f3008103ff0 2024-09-12T03:26:12.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3008113fd0 con 0x7f3008109120 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.041+0000 7f300ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30081142d0 con 0x7f30081049a0 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f3007e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f300819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f3007e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f300819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53476/0 (socket says 172.21.15.50:53476) 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f3007e00640 1 -- 172.21.15.50:0/3682673202 learned_addr learned my addr 172.21.15.50:0/3682673202 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f300ca00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3008109120 0x7f300819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f3007400640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30081049a0 0x7f300819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f300ca00640 1 -- 172.21.15.50:0/3682673202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30081049a0 msgr2=0x7f300819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f300ca00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30081049a0 0x7f300819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f300ca00640 1 -- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 msgr2=0x7f300819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f300ca00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f300819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.042+0000 7f300ca00640 1 -- 172.21.15.50:0/3682673202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300819f5e0 con 0x7f3008109120 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.043+0000 7f300ca00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3008109120 0x7f300819ef00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3000009fd0 tx=0x7f300000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.043+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300000ce20 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.043+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000005020 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.043+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300819f8d0 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.043+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000010720 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.043+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30081a71b0 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.044+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30081043d0 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.046+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3000005540 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.047+0000 7f3004c00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2fec0a2c10 0x7f2fec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.048+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f30000cfb20 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.048+0000 7f3007e00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2fec0a2c10 0x7f2fec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.049+0000 7f3007e00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2fec0a2c10 0x7f2fec0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80096c0 tx=0x7f2ff8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.051+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f300009c560 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.243+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f300802d7c0 con 0x7f3008109120 2024-09-12T03:26:12.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.243+0000 7f3004c00640 1 -- 172.21.15.50:0/3682673202 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f300009be60 con 0x7f3008109120 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stdout:77309411409 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2fec0a2c10 msgr2=0x7f2fec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2fec0a2c10 0x7f2fec0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80096c0 tx=0x7f2ff8005a20 comp rx=0 tx=0).stop 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3008109120 msgr2=0x7f300819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3008109120 0x7f300819ef00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3000009fd0 tx=0x7f300000ef10 comp rx=0 tx=0).stop 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 shutdown_connections 2024-09-12T03:26:12.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2fec0a2c10 0x7f2fec0a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3008109120 0x7f300819ef00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30081049a0 0x7f300819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 --2- 172.21.15.50:0/3682673202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3008103ff0 0x7f300819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 >> 172.21.15.50:0/3682673202 conn(0x7f30080ffa90 msgr2=0x7f3008100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 shutdown_connections 2024-09-12T03:26:12.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:10.246+0000 7f300ea00640 1 -- 172.21.15.50:0/3682673202 wait complete. 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 -- 172.21.15.50:0/1100658879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c4103ff0 msgr2=0x7f05c41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 --2- 172.21.15.50:0/1100658879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c4103ff0 0x7f05c41043d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009900 tx=0x7f05b402a410 comp rx=0 tx=0).stop 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 -- 172.21.15.50:0/1100658879 shutdown_connections 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 --2- 172.21.15.50:0/1100658879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 0x7f05c4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 --2- 172.21.15.50:0/1100658879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c41049a0 0x7f05c41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 --2- 172.21.15.50:0/1100658879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c4103ff0 0x7f05c41043d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 -- 172.21.15.50:0/1100658879 >> 172.21.15.50:0/1100658879 conn(0x7f05c40ffa90 msgr2=0x7f05c4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 -- 172.21.15.50:0/1100658879 shutdown_connections 2024-09-12T03:26:12.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.768+0000 7f05ca200640 1 -- 172.21.15.50:0/1100658879 wait complete. 2024-09-12T03:26:12.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 Processor -- start 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 -- start start 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c4103ff0 0x7f05c419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 0x7f05c419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 0x7f05c419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05c4114140 con 0x7f05c41049a0 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f05c4113fc0 con 0x7f05c4109120 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.769+0000 7f05ca200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05c41142c0 con 0x7f05c4103ff0 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 0x7f05c419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 0x7f05c419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53560/0 (socket says 172.21.15.50:53560) 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 -- 172.21.15.50:0/939664593 learned_addr learned my addr 172.21.15.50:0/939664593 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05c3400640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c4103ff0 0x7f05c419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05c3e00640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 0x7f05c419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 -- 172.21.15.50:0/939664593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c4103ff0 msgr2=0x7f05c419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c4103ff0 0x7f05c419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 -- 172.21.15.50:0/939664593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 msgr2=0x7f05c419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 0x7f05c419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05bbe00640 1 -- 172.21.15.50:0/939664593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c419f5d0 con 0x7f05c41049a0 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05c3400640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c4103ff0 0x7f05c419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.770+0000 7f05c3e00640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 0x7f05c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.771+0000 7f05bbe00640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 0x7f05c419ab60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00d770 tx=0x7f05ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.771+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac017070 con 0x7f05c41049a0 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.771+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05ac005500 con 0x7f05c41049a0 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.771+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c419f8c0 con 0x7f05c41049a0 2024-09-12T03:26:12.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.771+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac004e00 con 0x7f05c41049a0 2024-09-12T03:26:12.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.771+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c41a71a0 con 0x7f05c41049a0 2024-09-12T03:26:12.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.772+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c41043d0 con 0x7f05c41049a0 2024-09-12T03:26:12.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.775+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f05ac004fa0 con 0x7f05c41049a0 2024-09-12T03:26:12.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.776+0000 7f05c0c00640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05a80a2c10 0x7f05a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.777+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f05ac0cef10 con 0x7f05c41049a0 2024-09-12T03:26:12.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.777+0000 7f05c3400640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05a80a2c10 0x7f05a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.777+0000 7f05c3400640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05a80a2c10 0x7f05a80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009230 tx=0x7f05b40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.781+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05ac09b300 con 0x7f05c41049a0 2024-09-12T03:26:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:12 smithi050 ceph-mon[30378]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:26:12.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:12 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/233700534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:26:12.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.975+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f05c410bd70 con 0x7f05c41049a0 2024-09-12T03:26:12.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.976+0000 7f05c0c00640 1 -- 172.21.15.50:0/939664593 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f05ac09f600 con 0x7f05c41049a0 2024-09-12T03:26:12.976 INFO:teuthology.orchestra.run.smithi050.stdout:188978561084 2024-09-12T03:26:12.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05a80a2c10 msgr2=0x7f05a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05a80a2c10 0x7f05a80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009230 tx=0x7f05b40058d0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 msgr2=0x7f05c419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 0x7f05c419ab60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00d770 tx=0x7f05ac00dc30 comp rx=0 tx=0).stop 2024-09-12T03:26:12.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 shutdown_connections 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f05a80a2c10 0x7f05a80a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05c4109120 0x7f05c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05c41049a0 0x7f05c419ab60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 --2- 172.21.15.50:0/939664593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05c4103ff0 0x7f05c419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.979+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 >> 172.21.15.50:0/939664593 conn(0x7f05c40ffa90 msgr2=0x7f05c4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.980+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 shutdown_connections 2024-09-12T03:26:12.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.980+0000 7f05ca200640 1 -- 172.21.15.50:0/939664593 wait complete. 2024-09-12T03:26:12.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.990+0000 7f8673200640 1 -- 172.21.15.50:0/3417914397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 msgr2=0x7f86740749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.990+0000 7f8673200640 1 --2- 172.21.15.50:0/3417914397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f86740749a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8664009900 tx=0x7f866402a480 comp rx=0 tx=0).stop 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 -- 172.21.15.50:0/3417914397 shutdown_connections 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 --2- 172.21.15.50:0/3417914397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 0x7f8674110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 --2- 172.21.15.50:0/3417914397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 0x7f867410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 --2- 172.21.15.50:0/3417914397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f86740749a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 -- 172.21.15.50:0/3417914397 >> 172.21.15.50:0/3417914397 conn(0x7f86740fd740 msgr2=0x7f86740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 -- 172.21.15.50:0/3417914397 shutdown_connections 2024-09-12T03:26:12.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.991+0000 7f8673200640 1 -- 172.21.15.50:0/3417914397 wait complete. 2024-09-12T03:26:12.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 Processor -- start 2024-09-12T03:26:12.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 -- start start 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f867419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 0x7f867419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 0x7f867419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8674114000 con 0x7f86740745c0 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8674113e80 con 0x7f8674074ee0 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.992+0000 7f8673200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8674114180 con 0x7f867410e260 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8671e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f867419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8671e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f867419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53580/0 (socket says 172.21.15.50:53580) 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8671e00640 1 -- 172.21.15.50:0/4012721733 learned_addr learned my addr 172.21.15.50:0/4012721733 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8672800640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 0x7f867419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8671400640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 0x7f867419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8671e00640 1 -- 172.21.15.50:0/4012721733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 msgr2=0x7f867419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.993+0000 7f8671e00640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 0x7f867419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.994+0000 7f8671e00640 1 -- 172.21.15.50:0/4012721733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 msgr2=0x7f867419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.994+0000 7f8671e00640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 0x7f867419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.994+0000 7f8671e00640 1 -- 172.21.15.50:0/4012721733 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867419f490 con 0x7f86740745c0 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.994+0000 7f8672800640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 0x7f867419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:12.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.994+0000 7f8671400640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 0x7f867419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:12.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.994+0000 7f8671e00640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f867419a4e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8664004770 tx=0x7f8664003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.995+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866403f070 con 0x7f86740745c0 2024-09-12T03:26:12.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.995+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664031ca0 con 0x7f86740745c0 2024-09-12T03:26:12.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.995+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664035030 con 0x7f86740745c0 2024-09-12T03:26:12.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.995+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867419f720 con 0x7f86740745c0 2024-09-12T03:26:12.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.995+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86741a7060 con 0x7f86740745c0 2024-09-12T03:26:12.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.997+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86740749a0 con 0x7f86740745c0 2024-09-12T03:26:12.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:12.999+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8664042070 con 0x7f86740745c0 2024-09-12T03:26:13.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.000+0000 7f866aa00640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f86580a2c90 0x7f86580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:13.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.000+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f86640efa40 con 0x7f86740745c0 2024-09-12T03:26:13.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.000+0000 7f8671400640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f86580a2c90 0x7f86580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.001+0000 7f8671400640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f86580a2c90 0x7f86580a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86600095f0 tx=0x7f8660005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:13.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.004+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86640bbf50 con 0x7f86740745c0 2024-09-12T03:26:13.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:12 smithi142 ceph-mon[35329]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:26:13.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:12 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/233700534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:26:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:12 smithi082 ceph-mon[35046]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:26:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:12 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/233700534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:26:13.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.194+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f86741a7230 con 0x7f86740745c0 2024-09-12T03:26:13.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.194+0000 7f866aa00640 1 -- 172.21.15.50:0/4012721733 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f86640c0250 con 0x7f86740745c0 2024-09-12T03:26:13.194 INFO:teuthology.orchestra.run.smithi050.stdout:210453397559 2024-09-12T03:26:13.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f86580a2c90 msgr2=0x7f86580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f86580a2c90 0x7f86580a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86600095f0 tx=0x7f8660005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 msgr2=0x7f867419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f867419a4e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8664004770 tx=0x7f8664003c10 comp rx=0 tx=0).stop 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 shutdown_connections 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f86580a2c90 0x7f86580a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867410e260 0x7f867419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8674074ee0 0x7f867419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 --2- 172.21.15.50:0/4012721733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86740745c0 0x7f867419a4e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 >> 172.21.15.50:0/4012721733 conn(0x7f86740fd740 msgr2=0x7f86740fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 shutdown_connections 2024-09-12T03:26:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:13.197+0000 7f8673200640 1 -- 172.21.15.50:0/4012721733 wait complete. 2024-09-12T03:26:13.871 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:26:13.871 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-12T03:26:13.871 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:13 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T03:26:13.871 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:13 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/939664593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:26:13.871 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:13 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4012721733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:26:14.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:26:14.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-12T03:26:14.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:13 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T03:26:14.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:13 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/939664593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:26:14.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:13 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4012721733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:26:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:26:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-12T03:26:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:13 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T03:26:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:13 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/939664593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:26:14.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:13 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4012721733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 -- 172.21.15.50:0/4022991628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 msgr2=0x7fc5481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 --2- 172.21.15.50:0/4022991628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc5481043d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009900 tx=0x7fc53c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 -- 172.21.15.50:0/4022991628 shutdown_connections 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 --2- 172.21.15.50:0/4022991628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 0x7fc548110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 --2- 172.21.15.50:0/4022991628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5481049a0 0x7fc5481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 --2- 172.21.15.50:0/4022991628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc5481043d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 -- 172.21.15.50:0/4022991628 >> 172.21.15.50:0/4022991628 conn(0x7fc5480ffa90 msgr2=0x7fc548101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 -- 172.21.15.50:0/4022991628 shutdown_connections 2024-09-12T03:26:14.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.166+0000 7fc54f600640 1 -- 172.21.15.50:0/4022991628 wait complete. 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 Processor -- start 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 -- start start 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc54819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5481049a0 0x7fc54819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 0x7fc54819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc548114170 con 0x7fc548103ff0 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc548113ff0 con 0x7fc548109120 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.167+0000 7fc54f600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481142f0 con 0x7fc5481049a0 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc54819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc54819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53592/0 (socket says 172.21.15.50:53592) 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 -- 172.21.15.50:0/2663824557 learned_addr learned my addr 172.21.15.50:0/2663824557 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc547e00640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5481049a0 0x7fc54819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54d600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 0x7fc54819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 -- 172.21.15.50:0/2663824557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5481049a0 msgr2=0x7fc54819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5481049a0 0x7fc54819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 -- 172.21.15.50:0/2663824557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 msgr2=0x7fc54819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 0x7fc54819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54cc00640 1 -- 172.21.15.50:0/2663824557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54819f5e0 con 0x7fc548103ff0 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.168+0000 7fc54d600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 0x7fc54819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.169+0000 7fc54cc00640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc54819a630 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc53c004770 tx=0x7fc53c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.169+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c03f070 con 0x7fc548103ff0 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.169+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc53c031ca0 con 0x7fc548103ff0 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.169+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54819f870 con 0x7fc548103ff0 2024-09-12T03:26:14.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.169+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c035030 con 0x7fc548103ff0 2024-09-12T03:26:14.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.169+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a71b0 con 0x7fc548103ff0 2024-09-12T03:26:14.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.170+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5481054d0 con 0x7fc548103ff0 2024-09-12T03:26:14.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.172+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc53c042070 con 0x7fc548103ff0 2024-09-12T03:26:14.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.173+0000 7fc545600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5300a2c10 0x7fc5300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.174+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fc53c0efc70 con 0x7fc548103ff0 2024-09-12T03:26:14.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.174+0000 7fc547e00640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5300a2c10 0x7fc5300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.175+0000 7fc547e00640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5300a2c10 0x7fc5300a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5480fff90 tx=0x7fc534005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:14.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.178+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc53c0bc720 con 0x7fc548103ff0 2024-09-12T03:26:14.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.366+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc54810c300 con 0x7fc548103ff0 2024-09-12T03:26:14.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.366+0000 7fc545600640 1 -- 172.21.15.50:0/2663824557 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc53c0bc020 con 0x7fc548103ff0 2024-09-12T03:26:14.366 INFO:teuthology.orchestra.run.smithi050.stdout:158913790018 2024-09-12T03:26:14.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.368+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5300a2c10 msgr2=0x7fc5300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.368+0000 7fc54f600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5300a2c10 0x7fc5300a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5480fff90 tx=0x7fc534005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.368+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 msgr2=0x7fc54819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.368+0000 7fc54f600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc54819a630 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc53c004770 tx=0x7fc53c003c10 comp rx=0 tx=0).stop 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 shutdown_connections 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc5300a2c10 0x7fc5300a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc548109120 0x7fc54819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5481049a0 0x7fc54819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 --2- 172.21.15.50:0/2663824557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc548103ff0 0x7fc54819a630 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 >> 172.21.15.50:0/2663824557 conn(0x7fc5480ffa90 msgr2=0x7fc548100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 shutdown_connections 2024-09-12T03:26:14.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:14.369+0000 7fc54f600640 1 -- 172.21.15.50:0/2663824557 wait complete. 2024-09-12T03:26:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:14 smithi050 ceph-mon[30378]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-12T03:26:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:26:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-12T03:26:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:14 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T03:26:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:14 smithi050 ceph-mon[30378]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:26:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:14 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2663824557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:26:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:14 smithi142 ceph-mon[35329]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-12T03:26:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:26:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-12T03:26:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:14 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T03:26:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:14 smithi142 ceph-mon[35329]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:26:15.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:14 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2663824557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:26:15.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953542 got 137438953543 for osd.4 2024-09-12T03:26:15.064 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:14 smithi082 ceph-mon[35046]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-12T03:26:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:26:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-12T03:26:15.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:14 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T03:26:15.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:14 smithi082 ceph-mon[35046]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:26:15.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:14 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2663824557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 -- 172.21.15.50:0/2696769014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 msgr2=0x7ff740101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 --2- 172.21.15.50:0/2696769014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff740101e30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009930 tx=0x7ff72c02a420 comp rx=0 tx=0).stop 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 -- 172.21.15.50:0/2696769014 shutdown_connections 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 --2- 172.21.15.50:0/2696769014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740102370 0x7ff74010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 --2- 172.21.15.50:0/2696769014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff740101e30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 --2- 172.21.15.50:0/2696769014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7401079d0 0x7ff740107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 -- 172.21.15.50:0/2696769014 >> 172.21.15.50:0/2696769014 conn(0x7ff7400fd710 msgr2=0x7ff7400ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 -- 172.21.15.50:0/2696769014 shutdown_connections 2024-09-12T03:26:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.286+0000 7ff745200640 1 -- 172.21.15.50:0/2696769014 wait complete. 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.287+0000 7ff745200640 1 Processor -- start 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.287+0000 7ff745200640 1 -- start start 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff745200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff74019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff745200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff740102370 0x7ff74019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff745200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7401079d0 0x7ff74019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff745200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111e20 con 0x7ff7401019d0 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff745200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111ca0 con 0x7ff740102370 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff745200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111fa0 con 0x7ff7401079d0 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff73f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff74019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff73f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff74019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53614/0 (socket says 172.21.15.50:53614) 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff73fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7401079d0 0x7ff74019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff73f400640 1 -- 172.21.15.50:0/853396113 learned_addr learned my addr 172.21.15.50:0/853396113 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff73ea00640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff740102370 0x7ff74019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.288+0000 7ff73f400640 1 -- 172.21.15.50:0/853396113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7401079d0 msgr2=0x7ff74019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff73f400640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7401079d0 0x7ff74019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff73f400640 1 -- 172.21.15.50:0/853396113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff740102370 msgr2=0x7ff74019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff73f400640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff740102370 0x7ff74019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff73f400640 1 -- 172.21.15.50:0/853396113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74019f430 con 0x7ff7401019d0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff73f400640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff74019a480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9f0 tx=0x7ff73000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000cd80 con 0x7ff7401019d0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff730010910 con 0x7ff7401019d0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730018990 con 0x7ff7401019d0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74019f720 con 0x7ff7401019d0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.289+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401a7000 con 0x7ff7401019d0 2024-09-12T03:26:15.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.290+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff740101e30 con 0x7ff7401019d0 2024-09-12T03:26:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.292+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff730010ac0 con 0x7ff7401019d0 2024-09-12T03:26:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.294+0000 7ff737e00640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff7240a2c90 0x7ff7240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.294+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7ff7300cf7b0 con 0x7ff7401019d0 2024-09-12T03:26:15.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.294+0000 7ff73ea00640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff7240a2c90 0x7ff7240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.295+0000 7ff73ea00640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff7240a2c90 0x7ff7240a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff72c006010 tx=0x7ff72c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:15.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.297+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff73009bb50 con 0x7ff7401019d0 2024-09-12T03:26:15.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.486+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff7401a7230 con 0x7ff7401019d0 2024-09-12T03:26:15.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.487+0000 7ff737e00640 1 -- 172.21.15.50:0/853396113 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff73009fe50 con 0x7ff7401019d0 2024-09-12T03:26:15.487 INFO:teuthology.orchestra.run.smithi050.stdout:34359738460 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff7240a2c90 msgr2=0x7ff7240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff7240a2c90 0x7ff7240a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff72c006010 tx=0x7ff72c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 msgr2=0x7ff74019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff74019a480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9f0 tx=0x7ff73000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 shutdown_connections 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7ff7240a2c90 0x7ff7240a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7401079d0 0x7ff74019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff740102370 0x7ff74019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 --2- 172.21.15.50:0/853396113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7401019d0 0x7ff74019a480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.489+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 >> 172.21.15.50:0/853396113 conn(0x7ff7400fd710 msgr2=0x7ff7400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:15.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.490+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 shutdown_connections 2024-09-12T03:26:15.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.490+0000 7ff745200640 1 -- 172.21.15.50:0/853396113 wait complete. 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 -- 172.21.15.50:0/540230152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1068d0 msgr2=0x7f7e3c106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 --2- 172.21.15.50:0/540230152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c106cb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009930 tx=0x7f7e2c02a420 comp rx=0 tx=0).stop 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 -- 172.21.15.50:0/540230152 shutdown_connections 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 --2- 172.21.15.50:0/540230152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 0x7f7e3c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 --2- 172.21.15.50:0/540230152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 --2- 172.21.15.50:0/540230152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c106cb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 -- 172.21.15.50:0/540230152 >> 172.21.15.50:0/540230152 conn(0x7f7e3c0fc610 msgr2=0x7f7e3c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 -- 172.21.15.50:0/540230152 shutdown_connections 2024-09-12T03:26:15.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.511+0000 7f7e3b200640 1 -- 172.21.15.50:0/540230152 wait complete. 2024-09-12T03:26:15.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.514+0000 7f7e3b200640 1 Processor -- start 2024-09-12T03:26:15.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.514+0000 7f7e3b200640 1 -- start start 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 0x7f7e3c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c1a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c110d20 con 0x7f7e3c1008d0 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c110ba0 con 0x7f7e3c101270 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c110ea0 con 0x7f7e3c1068d0 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e39e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e39e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53632/0 (socket says 172.21.15.50:53632) 2024-09-12T03:26:15.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e39e00640 1 -- 172.21.15.50:0/3709460054 learned_addr learned my addr 172.21.15.50:0/3709460054 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:15.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e39400640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 0x7f7e3c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.515+0000 7f7e3a800640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c1a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39400640 1 -- 172.21.15.50:0/3709460054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1068d0 msgr2=0x7f7e3c1a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39400640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c1a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39400640 1 -- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 msgr2=0x7f7e3c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39400640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39400640 1 -- 172.21.15.50:0/3709460054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c1a46d0 con 0x7f7e3c101270 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e3a800640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c1a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39e00640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:15.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e39400640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 0x7f7e3c199650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e9f0 tx=0x7f7e2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:15.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.516+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800ccc0 con 0x7f7e3c101270 2024-09-12T03:26:15.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.517+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c1a48a0 con 0x7f7e3c101270 2024-09-12T03:26:15.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.517+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e28005020 con 0x7f7e3c101270 2024-09-12T03:26:15.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.517+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e280049e0 con 0x7f7e3c101270 2024-09-12T03:26:15.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.517+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a4db0 con 0x7f7e3c101270 2024-09-12T03:26:15.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.518+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c100d30 con 0x7f7e3c101270 2024-09-12T03:26:15.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.520+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e28005560 con 0x7f7e3c101270 2024-09-12T03:26:15.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.521+0000 7f7e32a00640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7e200a2c90 0x7f7e200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:15.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.522+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f7e280cf630 con 0x7f7e3c101270 2024-09-12T03:26:15.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.522+0000 7f7e39e00640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7e200a2c90 0x7f7e200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.522+0000 7f7e39e00640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7e200a2c90 0x7f7e200a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000c00 tx=0x7f7e2c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.523+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e280051d0 con 0x7f7e3c101270 2024-09-12T03:26:15.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.723+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7e3c02d7c0 con 0x7f7e3c101270 2024-09-12T03:26:15.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.724+0000 7f7e32a00640 1 -- 172.21.15.50:0/3709460054 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7e2809c0b0 con 0x7f7e3c101270 2024-09-12T03:26:15.724 INFO:teuthology.orchestra.run.smithi050.stdout:330712481826 2024-09-12T03:26:15.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.726+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7e200a2c90 msgr2=0x7f7e200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.726+0000 7f7e3b200640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7e200a2c90 0x7f7e200a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000c00 tx=0x7f7e2c031510 comp rx=0 tx=0).stop 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.726+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 msgr2=0x7f7e3c199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.726+0000 7f7e3b200640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 0x7f7e3c199650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e9f0 tx=0x7f7e2800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 shutdown_connections 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7e200a2c90 0x7f7e200a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e3c1068d0 0x7f7e3c1a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c101270 0x7f7e3c199650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 --2- 172.21.15.50:0/3709460054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1008d0 0x7f7e3c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 >> 172.21.15.50:0/3709460054 conn(0x7f7e3c0fc610 msgr2=0x7f7e3c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 shutdown_connections 2024-09-12T03:26:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:15.727+0000 7f7e3b200640 1 -- 172.21.15.50:0/3709460054 wait complete. 2024-09-12T03:26:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:15 smithi050 ceph-mon[30378]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:26:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:15 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/853396113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:26:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:15 smithi142 ceph-mon[35329]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:26:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:15 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/853396113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:26:16.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:15 smithi082 ceph-mon[35046]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:26:16.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:15 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/853396113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:26:16.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574931 got 55834574934 for osd.1 2024-09-12T03:26:16.566 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:16.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411407 got 77309411409 for osd.2 2024-09-12T03:26:16.904 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:17.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:16 smithi142 ceph-mon[35329]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:26:17.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:16 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3709460054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:26:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:16 smithi082 ceph-mon[35046]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:26:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:16 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3709460054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:26:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:16 smithi050 ceph-mon[30378]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:26:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:16 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3709460054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:26:17.554 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790016 got 158913790018 for osd.5 2024-09-12T03:26:17.554 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:17.913 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906985 got 274877906986 for osd.9 2024-09-12T03:26:17.913 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:18.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481822 got 330712481826 for osd.11 2024-09-12T03:26:18.436 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:18.844 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:18 smithi050 ceph-mon[30378]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:26:18.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168624 got 240518168627 for osd.8 2024-09-12T03:26:18.883 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:19.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:18 smithi142 ceph-mon[35329]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:26:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:18 smithi082 ceph-mon[35046]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:26:19.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738460 for osd.0 2024-09-12T03:26:19.160 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:19.992 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561080 got 188978561084 for osd.6 2024-09-12T03:26:19.992 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:20.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397557 got 210453397559 for osd.7 2024-09-12T03:26:20.393 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:20.819 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710756 got 300647710758 for osd.10 2024-09-12T03:26:20.819 DEBUG:teuthology.parallel:result is None 2024-09-12T03:26:20.819 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:26:20.820 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:26:21.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:20 smithi142 ceph-mon[35329]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:26:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:20 smithi082 ceph-mon[35046]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:26:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:20 smithi050 ceph-mon[30378]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:26:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:21.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 -- 172.21.15.50:0/2261880802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4103ff0 msgr2=0x7f1df41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 --2- 172.21.15.50:0/2261880802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4103ff0 0x7f1df41043d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009900 tx=0x7f1de402a410 comp rx=0 tx=0).stop 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 -- 172.21.15.50:0/2261880802 shutdown_connections 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 --2- 172.21.15.50:0/2261880802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4109120 0x7f1df4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 --2- 172.21.15.50:0/2261880802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df41049a0 0x7f1df41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 --2- 172.21.15.50:0/2261880802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4103ff0 0x7f1df41043d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 -- 172.21.15.50:0/2261880802 >> 172.21.15.50:0/2261880802 conn(0x7f1df40ffa90 msgr2=0x7f1df4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 -- 172.21.15.50:0/2261880802 shutdown_connections 2024-09-12T03:26:21.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.725+0000 7f1dfba00640 1 -- 172.21.15.50:0/2261880802 wait complete. 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.726+0000 7f1dfba00640 1 Processor -- start 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 -- start start 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 0x7f1df419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df41049a0 0x7f1df419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4109120 0x7f1df419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4114120 con 0x7f1df4109120 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4113fa0 con 0x7f1df41049a0 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1dfba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41142a0 con 0x7f1df4103ff0 2024-09-12T03:26:21.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1df9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 0x7f1df419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1df9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 0x7f1df419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45160/0 (socket says 172.21.15.50:45160) 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1df9000640 1 -- 172.21.15.50:0/1349346060 learned_addr learned my addr 172.21.15.50:0/1349346060 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1debe00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df41049a0 0x7f1df419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.727+0000 7f1df9a00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4109120 0x7f1df419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df9000640 1 -- 172.21.15.50:0/1349346060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df41049a0 msgr2=0x7f1df419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df9000640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df41049a0 0x7f1df419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df9000640 1 -- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4109120 msgr2=0x7f1df419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df9000640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4109120 0x7f1df419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df9000640 1 -- 172.21.15.50:0/1349346060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df419f5d0 con 0x7f1df4103ff0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df9000640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 0x7f1df419a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1de4006010 tx=0x7f1de40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de403f030 con 0x7f1df4103ff0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df419f860 con 0x7f1df4103ff0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de4003ce0 con 0x7f1df4103ff0 2024-09-12T03:26:21.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.728+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4035070 con 0x7f1df4103ff0 2024-09-12T03:26:21.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.729+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df41a71a0 con 0x7f1df4103ff0 2024-09-12T03:26:21.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.730+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df41043d0 con 0x7f1df4103ff0 2024-09-12T03:26:21.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.731+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1de4042030 con 0x7f1df4103ff0 2024-09-12T03:26:21.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.732+0000 7f1df2000640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1dd80a2c10 0x7f1dd80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:21.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.733+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1de40efb10 con 0x7f1df4103ff0 2024-09-12T03:26:21.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.733+0000 7f1debe00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1dd80a2c10 0x7f1dd80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:21.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.734+0000 7f1debe00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1dd80a2c10 0x7f1dd80a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1df40ffeb0 tx=0x7f1ddc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:21.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.737+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1de40bbf70 con 0x7f1df4103ff0 2024-09-12T03:26:21.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.919+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1df4062630 con 0x7f1dd80a2c10 2024-09-12T03:26:21.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.928+0000 7f1df2000640 1 -- 172.21.15.50:0/1349346060 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340372 (secure 0 0 0) 0x7f1df4062630 con 0x7f1dd80a2c10 2024-09-12T03:26:21.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:21.938 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-12T03:26:21.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1dd80a2c10 msgr2=0x7f1dd80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:21.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1dd80a2c10 0x7f1dd80a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1df40ffeb0 tx=0x7f1ddc005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:21.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 msgr2=0x7f1df419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:21.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 0x7f1df419a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1de4006010 tx=0x7f1de40042f0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 shutdown_connections 2024-09-12T03:26:21.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1dd80a2c10 0x7f1dd80a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1df4109120 0x7f1df419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df41049a0 0x7f1df419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 --2- 172.21.15.50:0/1349346060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df4103ff0 0x7f1df419a620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:21.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 >> 172.21.15.50:0/1349346060 conn(0x7f1df40ffa90 msgr2=0x7f1df4100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:21.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 shutdown_connections 2024-09-12T03:26:21.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:21.933+0000 7f1dfba00640 1 -- 172.21.15.50:0/1349346060 wait complete. 2024-09-12T03:26:22.708 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-12T03:26:20.772886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":499,"num_read_kb":373,"num_write":352,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180870,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376532,"kb_used_data":14720,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565292,"statfs":{"total":1151940427776,"available":1151554859008,"internally_reserved":0,"allocated":15073280,"data_stored":8416661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002374"},"pg_stats":[{"pgid":"2.1f","version":"92'9","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660847+0000","last_change":"2024-09-12T03:24:29.302864+0000","last_active":"2024-09-12T03:26:14.660847+0000","last_peered":"2024-09-12T03:26:14.660847+0000","last_clean":"2024-09-12T03:26:14.660847+0000","last_became_active":"2024-09-12T03:24:29.302551+0000","last_became_peered":"2024-09-12T03:24:29.302551+0000","last_unstale":"2024-09-12T03:26:14.660847+0000","last_undegraded":"2024-09-12T03:26:14.660847+0000","last_fullsized":"2024-09-12T03:26:14.660847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:10.151772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685643+0000","last_change":"2024-09-12T03:24:31.649557+0000","last_active":"2024-09-12T03:26:14.685643+0000","last_peered":"2024-09-12T03:26:14.685643+0000","last_clean":"2024-09-12T03:26:14.685643+0000","last_became_active":"2024-09-12T03:24:31.649082+0000","last_became_peered":"2024-09-12T03:24:31.649082+0000","last_unstale":"2024-09-12T03:26:14.685643+0000","last_undegraded":"2024-09-12T03:26:14.685643+0000","last_fullsized":"2024-09-12T03:26:14.685643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:54.245063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663273+0000","last_change":"2024-09-12T03:24:40.708670+0000","last_active":"2024-09-12T03:26:14.663273+0000","last_peered":"2024-09-12T03:26:14.663273+0000","last_clean":"2024-09-12T03:26:14.663273+0000","last_became_active":"2024-09-12T03:24:40.708264+0000","last_became_peered":"2024-09-12T03:24:40.708264+0000","last_unstale":"2024-09-12T03:26:14.663273+0000","last_undegraded":"2024-09-12T03:26:14.663273+0000","last_fullsized":"2024-09-12T03:26:14.663273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:00.670492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685839+0000","last_change":"2024-09-12T03:24:29.312254+0000","last_active":"2024-09-12T03:26:14.685839+0000","last_peered":"2024-09-12T03:26:14.685839+0000","last_clean":"2024-09-12T03:26:14.685839+0000","last_became_active":"2024-09-12T03:24:29.311842+0000","last_became_peered":"2024-09-12T03:24:29.311842+0000","last_unstale":"2024-09-12T03:26:14.685839+0000","last_undegraded":"2024-09-12T03:26:14.685839+0000","last_fullsized":"2024-09-12T03:26:14.685839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:43.298456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669016+0000","last_change":"2024-09-12T03:24:31.644330+0000","last_active":"2024-09-12T03:26:14.669016+0000","last_peered":"2024-09-12T03:26:14.669016+0000","last_clean":"2024-09-12T03:26:14.669016+0000","last_became_active":"2024-09-12T03:24:31.642904+0000","last_became_peered":"2024-09-12T03:24:31.642904+0000","last_unstale":"2024-09-12T03:26:14.669016+0000","last_undegraded":"2024-09-12T03:26:14.669016+0000","last_fullsized":"2024-09-12T03:26:14.669016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:14.304333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669082+0000","last_change":"2024-09-12T03:24:40.706682+0000","last_active":"2024-09-12T03:26:14.669082+0000","last_peered":"2024-09-12T03:26:14.669082+0000","last_clean":"2024-09-12T03:26:14.669082+0000","last_became_active":"2024-09-12T03:24:40.706143+0000","last_became_peered":"2024-09-12T03:24:40.706143+0000","last_unstale":"2024-09-12T03:26:14.669082+0000","last_undegraded":"2024-09-12T03:26:14.669082+0000","last_fullsized":"2024-09-12T03:26:14.669082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:05.146486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665732+0000","last_change":"2024-09-12T03:24:29.307621+0000","last_active":"2024-09-12T03:26:14.665732+0000","last_peered":"2024-09-12T03:26:14.665732+0000","last_clean":"2024-09-12T03:26:14.665732+0000","last_became_active":"2024-09-12T03:24:29.307258+0000","last_became_peered":"2024-09-12T03:24:29.307258+0000","last_unstale":"2024-09-12T03:26:14.665732+0000","last_undegraded":"2024-09-12T03:26:14.665732+0000","last_fullsized":"2024-09-12T03:26:14.665732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:03.513367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670424+0000","last_change":"2024-09-12T03:24:31.649972+0000","last_active":"2024-09-12T03:26:14.670424+0000","last_peered":"2024-09-12T03:26:14.670424+0000","last_clean":"2024-09-12T03:26:14.670424+0000","last_became_active":"2024-09-12T03:24:31.649693+0000","last_became_peered":"2024-09-12T03:24:31.649693+0000","last_unstale":"2024-09-12T03:26:14.670424+0000","last_undegraded":"2024-09-12T03:26:14.670424+0000","last_fullsized":"2024-09-12T03:26:14.670424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:10.502054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669572+0000","last_change":"2024-09-12T03:24:40.698995+0000","last_active":"2024-09-12T03:26:14.669572+0000","last_peered":"2024-09-12T03:26:14.669572+0000","last_clean":"2024-09-12T03:26:14.669572+0000","last_became_active":"2024-09-12T03:24:40.698094+0000","last_became_peered":"2024-09-12T03:24:40.698094+0000","last_unstale":"2024-09-12T03:26:14.669572+0000","last_undegraded":"2024-09-12T03:26:14.669572+0000","last_fullsized":"2024-09-12T03:26:14.669572+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:39.266209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.675457+0000","last_change":"2024-09-12T03:26:14.674955+0000","last_active":"2024-09-12T03:26:14.675457+0000","last_peered":"2024-09-12T03:26:14.675457+0000","last_clean":"2024-09-12T03:26:14.675457+0000","last_became_active":"2024-09-12T03:26:14.671862+0000","last_became_peered":"2024-09-12T03:26:14.671862+0000","last_unstale":"2024-09-12T03:26:14.675457+0000","last_undegraded":"2024-09-12T03:26:14.675457+0000","last_fullsized":"2024-09-12T03:26:14.675457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:57.427928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670977+0000","last_change":"2024-09-12T03:24:31.648735+0000","last_active":"2024-09-12T03:26:14.670977+0000","last_peered":"2024-09-12T03:26:14.670977+0000","last_clean":"2024-09-12T03:26:14.670977+0000","last_became_active":"2024-09-12T03:24:31.648416+0000","last_became_peered":"2024-09-12T03:24:31.648416+0000","last_unstale":"2024-09-12T03:26:14.670977+0000","last_undegraded":"2024-09-12T03:26:14.670977+0000","last_fullsized":"2024-09-12T03:26:14.670977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:24.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669151+0000","last_change":"2024-09-12T03:24:40.699941+0000","last_active":"2024-09-12T03:26:14.669151+0000","last_peered":"2024-09-12T03:26:14.669151+0000","last_clean":"2024-09-12T03:26:14.669151+0000","last_became_active":"2024-09-12T03:24:40.699126+0000","last_became_peered":"2024-09-12T03:24:40.699126+0000","last_unstale":"2024-09-12T03:26:14.669151+0000","last_undegraded":"2024-09-12T03:26:14.669151+0000","last_fullsized":"2024-09-12T03:26:14.669151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:24.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684307+0000","last_change":"2024-09-12T03:24:31.650650+0000","last_active":"2024-09-12T03:26:14.684307+0000","last_peered":"2024-09-12T03:26:14.684307+0000","last_clean":"2024-09-12T03:26:14.684307+0000","last_became_active":"2024-09-12T03:24:31.650279+0000","last_became_peered":"2024-09-12T03:24:31.650279+0000","last_unstale":"2024-09-12T03:26:14.684307+0000","last_undegraded":"2024-09-12T03:26:14.684307+0000","last_fullsized":"2024-09-12T03:26:14.684307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:49.860317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665270+0000","last_change":"2024-09-12T03:24:29.307426+0000","last_active":"2024-09-12T03:26:14.665270+0000","last_peered":"2024-09-12T03:26:14.665270+0000","last_clean":"2024-09-12T03:26:14.665270+0000","last_became_active":"2024-09-12T03:24:29.307119+0000","last_became_peered":"2024-09-12T03:24:29.307119+0000","last_unstale":"2024-09-12T03:26:14.665270+0000","last_undegraded":"2024-09-12T03:26:14.665270+0000","last_fullsized":"2024-09-12T03:26:14.665270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:48.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670467+0000","last_change":"2024-09-12T03:24:40.700292+0000","last_active":"2024-09-12T03:26:14.670467+0000","last_peered":"2024-09-12T03:26:14.670467+0000","last_clean":"2024-09-12T03:26:14.670467+0000","last_became_active":"2024-09-12T03:24:40.699819+0000","last_became_peered":"2024-09-12T03:24:40.699819+0000","last_unstale":"2024-09-12T03:26:14.670467+0000","last_undegraded":"2024-09-12T03:26:14.670467+0000","last_fullsized":"2024-09-12T03:26:14.670467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:43.022322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685437+0000","last_change":"2024-09-12T03:24:31.647669+0000","last_active":"2024-09-12T03:26:14.685437+0000","last_peered":"2024-09-12T03:26:14.685437+0000","last_clean":"2024-09-12T03:26:14.685437+0000","last_became_active":"2024-09-12T03:24:31.644429+0000","last_became_peered":"2024-09-12T03:24:31.644429+0000","last_unstale":"2024-09-12T03:26:14.685437+0000","last_undegraded":"2024-09-12T03:26:14.685437+0000","last_fullsized":"2024-09-12T03:26:14.685437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:22.556399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663435+0000","last_change":"2024-09-12T03:24:29.309334+0000","last_active":"2024-09-12T03:26:14.663435+0000","last_peered":"2024-09-12T03:26:14.663435+0000","last_clean":"2024-09-12T03:26:14.663435+0000","last_became_active":"2024-09-12T03:24:29.309007+0000","last_became_peered":"2024-09-12T03:24:29.309007+0000","last_unstale":"2024-09-12T03:26:14.663435+0000","last_undegraded":"2024-09-12T03:26:14.663435+0000","last_fullsized":"2024-09-12T03:26:14.663435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:40.898104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669668+0000","last_change":"2024-09-12T03:24:40.706611+0000","last_active":"2024-09-12T03:26:14.669668+0000","last_peered":"2024-09-12T03:26:14.669668+0000","last_clean":"2024-09-12T03:26:14.669668+0000","last_became_active":"2024-09-12T03:24:40.706058+0000","last_became_peered":"2024-09-12T03:24:40.706058+0000","last_unstale":"2024-09-12T03:26:14.669668+0000","last_undegraded":"2024-09-12T03:26:14.669668+0000","last_fullsized":"2024-09-12T03:26:14.669668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:16.547894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667191+0000","last_change":"2024-09-12T03:24:31.648458+0000","last_active":"2024-09-12T03:26:14.667191+0000","last_peered":"2024-09-12T03:26:14.667191+0000","last_clean":"2024-09-12T03:26:14.667191+0000","last_became_active":"2024-09-12T03:24:31.644171+0000","last_became_peered":"2024-09-12T03:24:31.644171+0000","last_unstale":"2024-09-12T03:26:14.667191+0000","last_undegraded":"2024-09-12T03:26:14.667191+0000","last_fullsized":"2024-09-12T03:26:14.667191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:20.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685145+0000","last_change":"2024-09-12T03:24:29.309013+0000","last_active":"2024-09-12T03:26:14.685145+0000","last_peered":"2024-09-12T03:26:14.685145+0000","last_clean":"2024-09-12T03:26:14.685145+0000","last_became_active":"2024-09-12T03:24:29.308459+0000","last_became_peered":"2024-09-12T03:24:29.308459+0000","last_unstale":"2024-09-12T03:26:14.685145+0000","last_undegraded":"2024-09-12T03:26:14.685145+0000","last_fullsized":"2024-09-12T03:26:14.685145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:38.655615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665514+0000","last_change":"2024-09-12T03:25:14.321079+0000","last_active":"2024-09-12T03:26:14.665514+0000","last_peered":"2024-09-12T03:26:14.665514+0000","last_clean":"2024-09-12T03:26:14.665514+0000","last_became_active":"2024-09-12T03:25:14.320602+0000","last_became_peered":"2024-09-12T03:25:14.320602+0000","last_unstale":"2024-09-12T03:26:14.665514+0000","last_undegraded":"2024-09-12T03:26:14.665514+0000","last_fullsized":"2024-09-12T03:26:14.665514+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:18.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665017+0000","last_change":"2024-09-12T03:24:31.637560+0000","last_active":"2024-09-12T03:26:14.665017+0000","last_peered":"2024-09-12T03:26:14.665017+0000","last_clean":"2024-09-12T03:26:14.665017+0000","last_became_active":"2024-09-12T03:24:31.636962+0000","last_became_peered":"2024-09-12T03:24:31.636962+0000","last_unstale":"2024-09-12T03:26:14.665017+0000","last_undegraded":"2024-09-12T03:26:14.665017+0000","last_fullsized":"2024-09-12T03:26:14.665017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:44.468397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670183+0000","last_change":"2024-09-12T03:24:29.305262+0000","last_active":"2024-09-12T03:26:14.670183+0000","last_peered":"2024-09-12T03:26:14.670183+0000","last_clean":"2024-09-12T03:26:14.670183+0000","last_became_active":"2024-09-12T03:24:29.304937+0000","last_became_peered":"2024-09-12T03:24:29.304937+0000","last_unstale":"2024-09-12T03:26:14.670183+0000","last_undegraded":"2024-09-12T03:26:14.670183+0000","last_fullsized":"2024-09-12T03:26:14.670183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:45.172526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670378+0000","last_change":"2024-09-12T03:24:40.702517+0000","last_active":"2024-09-12T03:26:14.670378+0000","last_peered":"2024-09-12T03:26:14.670378+0000","last_clean":"2024-09-12T03:26:14.670378+0000","last_became_active":"2024-09-12T03:24:40.702093+0000","last_became_peered":"2024-09-12T03:24:40.702093+0000","last_unstale":"2024-09-12T03:26:14.670378+0000","last_undegraded":"2024-09-12T03:26:14.670378+0000","last_fullsized":"2024-09-12T03:26:14.670378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:00.817214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668754+0000","last_change":"2024-09-12T03:24:31.650541+0000","last_active":"2024-09-12T03:26:14.668754+0000","last_peered":"2024-09-12T03:26:14.668754+0000","last_clean":"2024-09-12T03:26:14.668754+0000","last_became_active":"2024-09-12T03:24:31.650180+0000","last_became_peered":"2024-09-12T03:24:31.650180+0000","last_unstale":"2024-09-12T03:26:14.668754+0000","last_undegraded":"2024-09-12T03:26:14.668754+0000","last_fullsized":"2024-09-12T03:26:14.668754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:11.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670065+0000","last_change":"2024-09-12T03:24:29.335888+0000","last_active":"2024-09-12T03:26:14.670065+0000","last_peered":"2024-09-12T03:26:14.670065+0000","last_clean":"2024-09-12T03:26:14.670065+0000","last_became_active":"2024-09-12T03:24:29.335535+0000","last_became_peered":"2024-09-12T03:24:29.335535+0000","last_unstale":"2024-09-12T03:26:14.670065+0000","last_undegraded":"2024-09-12T03:26:14.670065+0000","last_fullsized":"2024-09-12T03:26:14.670065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:19.246874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663288+0000","last_change":"2024-09-12T03:24:40.696385+0000","last_active":"2024-09-12T03:26:14.663288+0000","last_peered":"2024-09-12T03:26:14.663288+0000","last_clean":"2024-09-12T03:26:14.663288+0000","last_became_active":"2024-09-12T03:24:40.695968+0000","last_became_peered":"2024-09-12T03:24:40.695968+0000","last_unstale":"2024-09-12T03:26:14.663288+0000","last_undegraded":"2024-09-12T03:26:14.663288+0000","last_fullsized":"2024-09-12T03:26:14.663288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:31.768769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.686005+0000","last_change":"2024-09-12T03:24:31.655017+0000","last_active":"2024-09-12T03:26:14.686005+0000","last_peered":"2024-09-12T03:26:14.686005+0000","last_clean":"2024-09-12T03:26:14.686005+0000","last_became_active":"2024-09-12T03:24:31.654597+0000","last_became_peered":"2024-09-12T03:24:31.654597+0000","last_unstale":"2024-09-12T03:26:14.686005+0000","last_undegraded":"2024-09-12T03:26:14.686005+0000","last_fullsized":"2024-09-12T03:26:14.686005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:13.053238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660720+0000","last_change":"2024-09-12T03:24:29.303991+0000","last_active":"2024-09-12T03:26:14.660720+0000","last_peered":"2024-09-12T03:26:14.660720+0000","last_clean":"2024-09-12T03:26:14.660720+0000","last_became_active":"2024-09-12T03:24:29.303682+0000","last_became_peered":"2024-09-12T03:24:29.303682+0000","last_unstale":"2024-09-12T03:26:14.660720+0000","last_undegraded":"2024-09-12T03:26:14.660720+0000","last_fullsized":"2024-09-12T03:26:14.660720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:59.403461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684749+0000","last_change":"2024-09-12T03:24:40.704304+0000","last_active":"2024-09-12T03:26:14.684749+0000","last_peered":"2024-09-12T03:26:14.684749+0000","last_clean":"2024-09-12T03:26:14.684749+0000","last_became_active":"2024-09-12T03:24:40.703967+0000","last_became_peered":"2024-09-12T03:24:40.703967+0000","last_unstale":"2024-09-12T03:26:14.684749+0000","last_undegraded":"2024-09-12T03:26:14.684749+0000","last_fullsized":"2024-09-12T03:26:14.684749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:00.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684960+0000","last_change":"2024-09-12T03:24:31.650303+0000","last_active":"2024-09-12T03:26:14.684960+0000","last_peered":"2024-09-12T03:26:14.684960+0000","last_clean":"2024-09-12T03:26:14.684960+0000","last_became_active":"2024-09-12T03:24:31.649223+0000","last_became_peered":"2024-09-12T03:24:31.649223+0000","last_unstale":"2024-09-12T03:26:14.684960+0000","last_undegraded":"2024-09-12T03:26:14.684960+0000","last_fullsized":"2024-09-12T03:26:14.684960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:45.190363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665953+0000","last_change":"2024-09-12T03:24:29.310570+0000","last_active":"2024-09-12T03:26:14.665953+0000","last_peered":"2024-09-12T03:26:14.665953+0000","last_clean":"2024-09-12T03:26:14.665953+0000","last_became_active":"2024-09-12T03:24:29.310249+0000","last_became_peered":"2024-09-12T03:24:29.310249+0000","last_unstale":"2024-09-12T03:26:14.665953+0000","last_undegraded":"2024-09-12T03:26:14.665953+0000","last_fullsized":"2024-09-12T03:26:14.665953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.049566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667328+0000","last_change":"2024-09-12T03:24:40.696716+0000","last_active":"2024-09-12T03:26:14.667328+0000","last_peered":"2024-09-12T03:26:14.667328+0000","last_clean":"2024-09-12T03:26:14.667328+0000","last_became_active":"2024-09-12T03:24:40.696322+0000","last_became_peered":"2024-09-12T03:24:40.696322+0000","last_unstale":"2024-09-12T03:26:14.667328+0000","last_undegraded":"2024-09-12T03:26:14.667328+0000","last_fullsized":"2024-09-12T03:26:14.667328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:31.315393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663998+0000","last_change":"2024-09-12T03:24:31.652360+0000","last_active":"2024-09-12T03:26:14.663998+0000","last_peered":"2024-09-12T03:26:14.663998+0000","last_clean":"2024-09-12T03:26:14.663998+0000","last_became_active":"2024-09-12T03:24:31.652066+0000","last_became_peered":"2024-09-12T03:24:31.652066+0000","last_unstale":"2024-09-12T03:26:14.663998+0000","last_undegraded":"2024-09-12T03:26:14.663998+0000","last_fullsized":"2024-09-12T03:26:14.663998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:54:15.312876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660586+0000","last_change":"2024-09-12T03:24:29.305811+0000","last_active":"2024-09-12T03:26:14.660586+0000","last_peered":"2024-09-12T03:26:14.660586+0000","last_clean":"2024-09-12T03:26:14.660586+0000","last_became_active":"2024-09-12T03:24:29.305531+0000","last_became_peered":"2024-09-12T03:24:29.305531+0000","last_unstale":"2024-09-12T03:26:14.660586+0000","last_undegraded":"2024-09-12T03:26:14.660586+0000","last_fullsized":"2024-09-12T03:26:14.660586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:08.496670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665042+0000","last_change":"2024-09-12T03:25:14.267079+0000","last_active":"2024-09-12T03:26:14.665042+0000","last_peered":"2024-09-12T03:26:14.665042+0000","last_clean":"2024-09-12T03:26:14.665042+0000","last_became_active":"2024-09-12T03:25:14.266625+0000","last_became_peered":"2024-09-12T03:25:14.266625+0000","last_unstale":"2024-09-12T03:26:14.665042+0000","last_undegraded":"2024-09-12T03:26:14.665042+0000","last_fullsized":"2024-09-12T03:26:14.665042+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663729+0000","last_change":"2024-09-12T03:25:13.812997+0000","last_active":"2024-09-12T03:26:14.663729+0000","last_peered":"2024-09-12T03:26:14.663729+0000","last_clean":"2024-09-12T03:26:14.663729+0000","last_became_active":"2024-09-12T03:25:13.812596+0000","last_became_peered":"2024-09-12T03:25:13.812596+0000","last_unstale":"2024-09-12T03:26:14.663729+0000","last_undegraded":"2024-09-12T03:26:14.663729+0000","last_fullsized":"2024-09-12T03:26:14.663729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:43.228416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684433+0000","last_change":"2024-09-12T03:24:29.301269+0000","last_active":"2024-09-12T03:26:14.684433+0000","last_peered":"2024-09-12T03:26:14.684433+0000","last_clean":"2024-09-12T03:26:14.684433+0000","last_became_active":"2024-09-12T03:24:29.300937+0000","last_became_peered":"2024-09-12T03:24:29.300937+0000","last_unstale":"2024-09-12T03:26:14.684433+0000","last_undegraded":"2024-09-12T03:26:14.684433+0000","last_fullsized":"2024-09-12T03:26:14.684433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:24.453709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684828+0000","last_change":"2024-09-12T03:24:40.708631+0000","last_active":"2024-09-12T03:26:14.684828+0000","last_peered":"2024-09-12T03:26:14.684828+0000","last_clean":"2024-09-12T03:26:14.684828+0000","last_became_active":"2024-09-12T03:24:40.706381+0000","last_became_peered":"2024-09-12T03:24:40.706381+0000","last_unstale":"2024-09-12T03:26:14.684828+0000","last_undegraded":"2024-09-12T03:26:14.684828+0000","last_fullsized":"2024-09-12T03:26:14.684828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:10.945538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670344+0000","last_change":"2024-09-12T03:24:31.652471+0000","last_active":"2024-09-12T03:26:14.670344+0000","last_peered":"2024-09-12T03:26:14.670344+0000","last_clean":"2024-09-12T03:26:14.670344+0000","last_became_active":"2024-09-12T03:24:31.651566+0000","last_became_peered":"2024-09-12T03:24:31.651566+0000","last_unstale":"2024-09-12T03:26:14.670344+0000","last_undegraded":"2024-09-12T03:26:14.670344+0000","last_fullsized":"2024-09-12T03:26:14.670344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:01.238554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":98,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:15.943456+0000","last_change":"2024-09-12T03:24:29.306214+0000","last_active":"2024-09-12T03:26:15.943456+0000","last_peered":"2024-09-12T03:26:15.943456+0000","last_clean":"2024-09-12T03:26:15.943456+0000","last_became_active":"2024-09-12T03:24:29.305846+0000","last_became_peered":"2024-09-12T03:24:29.305846+0000","last_unstale":"2024-09-12T03:26:15.943456+0000","last_undegraded":"2024-09-12T03:26:15.943456+0000","last_fullsized":"2024-09-12T03:26:15.943456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:50.940377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685256+0000","last_change":"2024-09-12T03:24:40.709412+0000","last_active":"2024-09-12T03:26:14.685256+0000","last_peered":"2024-09-12T03:26:14.685256+0000","last_clean":"2024-09-12T03:26:14.685256+0000","last_became_active":"2024-09-12T03:24:40.705106+0000","last_became_peered":"2024-09-12T03:24:40.705106+0000","last_unstale":"2024-09-12T03:26:14.685256+0000","last_undegraded":"2024-09-12T03:26:14.685256+0000","last_fullsized":"2024-09-12T03:26:14.685256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:32.963060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684523+0000","last_change":"2024-09-12T03:24:31.644654+0000","last_active":"2024-09-12T03:26:14.684523+0000","last_peered":"2024-09-12T03:26:14.684523+0000","last_clean":"2024-09-12T03:26:14.684523+0000","last_became_active":"2024-09-12T03:24:31.642477+0000","last_became_peered":"2024-09-12T03:24:31.642477+0000","last_unstale":"2024-09-12T03:26:14.684523+0000","last_undegraded":"2024-09-12T03:26:14.684523+0000","last_fullsized":"2024-09-12T03:26:14.684523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.758784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684214+0000","last_change":"2024-09-12T03:24:29.306850+0000","last_active":"2024-09-12T03:26:14.684214+0000","last_peered":"2024-09-12T03:26:14.684214+0000","last_clean":"2024-09-12T03:26:14.684214+0000","last_became_active":"2024-09-12T03:24:29.305902+0000","last_became_peered":"2024-09-12T03:24:29.305902+0000","last_unstale":"2024-09-12T03:26:14.684214+0000","last_undegraded":"2024-09-12T03:26:14.684214+0000","last_fullsized":"2024-09-12T03:26:14.684214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:02.564521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670180+0000","last_change":"2024-09-12T03:24:40.700966+0000","last_active":"2024-09-12T03:26:14.670180+0000","last_peered":"2024-09-12T03:26:14.670180+0000","last_clean":"2024-09-12T03:26:14.670180+0000","last_became_active":"2024-09-12T03:24:40.700553+0000","last_became_peered":"2024-09-12T03:24:40.700553+0000","last_unstale":"2024-09-12T03:26:14.670180+0000","last_undegraded":"2024-09-12T03:26:14.670180+0000","last_fullsized":"2024-09-12T03:26:14.670180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:14.855459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685372+0000","last_change":"2024-09-12T03:24:31.653431+0000","last_active":"2024-09-12T03:26:14.685372+0000","last_peered":"2024-09-12T03:26:14.685372+0000","last_clean":"2024-09-12T03:26:14.685372+0000","last_became_active":"2024-09-12T03:24:31.648771+0000","last_became_peered":"2024-09-12T03:24:31.648771+0000","last_unstale":"2024-09-12T03:26:14.685372+0000","last_undegraded":"2024-09-12T03:26:14.685372+0000","last_fullsized":"2024-09-12T03:26:14.685372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:54.052828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669709+0000","last_change":"2024-09-12T03:24:29.305825+0000","last_active":"2024-09-12T03:26:14.669709+0000","last_peered":"2024-09-12T03:26:14.669709+0000","last_clean":"2024-09-12T03:26:14.669709+0000","last_became_active":"2024-09-12T03:24:29.305468+0000","last_became_peered":"2024-09-12T03:24:29.305468+0000","last_unstale":"2024-09-12T03:26:14.669709+0000","last_undegraded":"2024-09-12T03:26:14.669709+0000","last_fullsized":"2024-09-12T03:26:14.669709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:30.899307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":368,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684290+0000","last_change":"2024-09-12T03:23:13.507232+0000","last_active":"2024-09-12T03:26:14.684290+0000","last_peered":"2024-09-12T03:26:14.684290+0000","last_clean":"2024-09-12T03:26:14.684290+0000","last_became_active":"2024-09-12T03:23:13.506683+0000","last_became_peered":"2024-09-12T03:23:13.506683+0000","last_unstale":"2024-09-12T03:26:14.684290+0000","last_undegraded":"2024-09-12T03:26:14.684290+0000","last_fullsized":"2024-09-12T03:26:14.684290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_clean_scrub_stamp":"2024-09-12T03:19:33.594997+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.684658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684664+0000","last_change":"2024-09-12T03:24:40.696812+0000","last_active":"2024-09-12T03:26:14.684664+0000","last_peered":"2024-09-12T03:26:14.684664+0000","last_clean":"2024-09-12T03:26:14.684664+0000","last_became_active":"2024-09-12T03:24:40.695586+0000","last_became_peered":"2024-09-12T03:24:40.695586+0000","last_unstale":"2024-09-12T03:26:14.684664+0000","last_undegraded":"2024-09-12T03:26:14.684664+0000","last_fullsized":"2024-09-12T03:26:14.684664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:10.086113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684668+0000","last_change":"2024-09-12T03:24:31.644517+0000","last_active":"2024-09-12T03:26:14.684668+0000","last_peered":"2024-09-12T03:26:14.684668+0000","last_clean":"2024-09-12T03:26:14.684668+0000","last_became_active":"2024-09-12T03:24:31.643996+0000","last_became_peered":"2024-09-12T03:24:31.643996+0000","last_unstale":"2024-09-12T03:26:14.684668+0000","last_undegraded":"2024-09-12T03:26:14.684668+0000","last_fullsized":"2024-09-12T03:26:14.684668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:57.543397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669418+0000","last_change":"2024-09-12T03:25:14.266550+0000","last_active":"2024-09-12T03:26:14.669418+0000","last_peered":"2024-09-12T03:26:14.669418+0000","last_clean":"2024-09-12T03:26:14.669418+0000","last_became_active":"2024-09-12T03:25:14.266202+0000","last_became_peered":"2024-09-12T03:25:14.266202+0000","last_unstale":"2024-09-12T03:26:14.669418+0000","last_undegraded":"2024-09-12T03:26:14.669418+0000","last_fullsized":"2024-09-12T03:26:14.669418+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:00.981147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670158+0000","last_change":"2024-09-12T03:24:40.697001+0000","last_active":"2024-09-12T03:26:14.670158+0000","last_peered":"2024-09-12T03:26:14.670158+0000","last_clean":"2024-09-12T03:26:14.670158+0000","last_became_active":"2024-09-12T03:24:40.696091+0000","last_became_peered":"2024-09-12T03:24:40.696091+0000","last_unstale":"2024-09-12T03:26:14.670158+0000","last_undegraded":"2024-09-12T03:26:14.670158+0000","last_fullsized":"2024-09-12T03:26:14.670158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:07.073034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684396+0000","last_change":"2024-09-12T03:24:31.655819+0000","last_active":"2024-09-12T03:26:14.684396+0000","last_peered":"2024-09-12T03:26:14.684396+0000","last_clean":"2024-09-12T03:26:14.684396+0000","last_became_active":"2024-09-12T03:24:31.655417+0000","last_became_peered":"2024-09-12T03:24:31.655417+0000","last_unstale":"2024-09-12T03:26:14.684396+0000","last_undegraded":"2024-09-12T03:26:14.684396+0000","last_fullsized":"2024-09-12T03:26:14.684396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:54.380047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670788+0000","last_change":"2024-09-12T03:24:29.303732+0000","last_active":"2024-09-12T03:26:14.670788+0000","last_peered":"2024-09-12T03:26:14.670788+0000","last_clean":"2024-09-12T03:26:14.670788+0000","last_became_active":"2024-09-12T03:24:29.302314+0000","last_became_peered":"2024-09-12T03:24:29.302314+0000","last_unstale":"2024-09-12T03:26:14.670788+0000","last_undegraded":"2024-09-12T03:26:14.670788+0000","last_fullsized":"2024-09-12T03:26:14.670788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:51.555510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668153+0000","last_change":"2024-09-12T03:25:13.817846+0000","last_active":"2024-09-12T03:26:14.668153+0000","last_peered":"2024-09-12T03:26:14.668153+0000","last_clean":"2024-09-12T03:26:14.668153+0000","last_became_active":"2024-09-12T03:25:13.817102+0000","last_became_peered":"2024-09-12T03:25:13.817102+0000","last_unstale":"2024-09-12T03:26:14.668153+0000","last_undegraded":"2024-09-12T03:26:14.668153+0000","last_fullsized":"2024-09-12T03:26:14.668153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:26.472731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684558+0000","last_change":"2024-09-12T03:24:31.645293+0000","last_active":"2024-09-12T03:26:14.684558+0000","last_peered":"2024-09-12T03:26:14.684558+0000","last_clean":"2024-09-12T03:26:14.684558+0000","last_became_active":"2024-09-12T03:24:31.644944+0000","last_became_peered":"2024-09-12T03:24:31.644944+0000","last_unstale":"2024-09-12T03:26:14.684558+0000","last_undegraded":"2024-09-12T03:26:14.684558+0000","last_fullsized":"2024-09-12T03:26:14.684558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:38.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685463+0000","last_change":"2024-09-12T03:24:29.302710+0000","last_active":"2024-09-12T03:26:14.685463+0000","last_peered":"2024-09-12T03:26:14.685463+0000","last_clean":"2024-09-12T03:26:14.685463+0000","last_became_active":"2024-09-12T03:24:29.302262+0000","last_became_peered":"2024-09-12T03:24:29.302262+0000","last_unstale":"2024-09-12T03:26:14.685463+0000","last_undegraded":"2024-09-12T03:26:14.685463+0000","last_fullsized":"2024-09-12T03:26:14.685463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:08.392390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669977+0000","last_change":"2024-09-12T03:24:40.696164+0000","last_active":"2024-09-12T03:26:14.669977+0000","last_peered":"2024-09-12T03:26:14.669977+0000","last_clean":"2024-09-12T03:26:14.669977+0000","last_became_active":"2024-09-12T03:24:40.695786+0000","last_became_peered":"2024-09-12T03:24:40.695786+0000","last_unstale":"2024-09-12T03:26:14.669977+0000","last_undegraded":"2024-09-12T03:26:14.669977+0000","last_fullsized":"2024-09-12T03:26:14.669977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:27.063196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667558+0000","last_change":"2024-09-12T03:24:31.645341+0000","last_active":"2024-09-12T03:26:14.667558+0000","last_peered":"2024-09-12T03:26:14.667558+0000","last_clean":"2024-09-12T03:26:14.667558+0000","last_became_active":"2024-09-12T03:24:31.644496+0000","last_became_peered":"2024-09-12T03:24:31.644496+0000","last_unstale":"2024-09-12T03:26:14.667558+0000","last_undegraded":"2024-09-12T03:26:14.667558+0000","last_fullsized":"2024-09-12T03:26:14.667558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:09.416806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.679152+0000","last_change":"2024-09-12T03:26:14.678640+0000","last_active":"2024-09-12T03:26:14.679152+0000","last_peered":"2024-09-12T03:26:14.679152+0000","last_clean":"2024-09-12T03:26:14.679152+0000","last_became_active":"2024-09-12T03:26:14.671800+0000","last_became_peered":"2024-09-12T03:26:14.671800+0000","last_unstale":"2024-09-12T03:26:14.679152+0000","last_undegraded":"2024-09-12T03:26:14.679152+0000","last_fullsized":"2024-09-12T03:26:14.679152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:40.858472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670432+0000","last_change":"2024-09-12T03:24:40.703795+0000","last_active":"2024-09-12T03:26:14.670432+0000","last_peered":"2024-09-12T03:26:14.670432+0000","last_clean":"2024-09-12T03:26:14.670432+0000","last_became_active":"2024-09-12T03:24:40.703361+0000","last_became_peered":"2024-09-12T03:24:40.703361+0000","last_unstale":"2024-09-12T03:26:14.670432+0000","last_undegraded":"2024-09-12T03:26:14.670432+0000","last_fullsized":"2024-09-12T03:26:14.670432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:12.430431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670311+0000","last_change":"2024-09-12T03:24:31.649521+0000","last_active":"2024-09-12T03:26:14.670311+0000","last_peered":"2024-09-12T03:26:14.670311+0000","last_clean":"2024-09-12T03:26:14.670311+0000","last_became_active":"2024-09-12T03:24:31.649217+0000","last_became_peered":"2024-09-12T03:24:31.649217+0000","last_unstale":"2024-09-12T03:26:14.670311+0000","last_undegraded":"2024-09-12T03:26:14.670311+0000","last_fullsized":"2024-09-12T03:26:14.670311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:33.885153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":79,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685820+0000","last_change":"2024-09-12T03:26:14.685820+0000","last_active":"2024-09-12T03:26:14.685820+0000","last_peered":"2024-09-12T03:26:14.685820+0000","last_clean":"2024-09-12T03:26:14.685820+0000","last_became_active":"2024-09-12T03:26:14.672467+0000","last_became_peered":"2024-09-12T03:26:14.672467+0000","last_unstale":"2024-09-12T03:26:14.685820+0000","last_undegraded":"2024-09-12T03:26:14.685820+0000","last_fullsized":"2024-09-12T03:26:14.685820+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:58.530737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685256+0000","last_change":"2024-09-12T03:24:40.707719+0000","last_active":"2024-09-12T03:26:14.685256+0000","last_peered":"2024-09-12T03:26:14.685256+0000","last_clean":"2024-09-12T03:26:14.685256+0000","last_became_active":"2024-09-12T03:24:40.707364+0000","last_became_peered":"2024-09-12T03:24:40.707364+0000","last_unstale":"2024-09-12T03:26:14.685256+0000","last_undegraded":"2024-09-12T03:26:14.685256+0000","last_fullsized":"2024-09-12T03:26:14.685256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:00.423368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684093+0000","last_change":"2024-09-12T03:24:31.646006+0000","last_active":"2024-09-12T03:26:14.684093+0000","last_peered":"2024-09-12T03:26:14.684093+0000","last_clean":"2024-09-12T03:26:14.684093+0000","last_became_active":"2024-09-12T03:24:31.643165+0000","last_became_peered":"2024-09-12T03:24:31.643165+0000","last_unstale":"2024-09-12T03:26:14.684093+0000","last_undegraded":"2024-09-12T03:26:14.684093+0000","last_fullsized":"2024-09-12T03:26:14.684093+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":96,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669870+0000","last_change":"2024-09-12T03:24:29.304521+0000","last_active":"2024-09-12T03:26:14.669870+0000","last_peered":"2024-09-12T03:26:14.669870+0000","last_clean":"2024-09-12T03:26:14.669870+0000","last_became_active":"2024-09-12T03:24:29.304147+0000","last_became_peered":"2024-09-12T03:24:29.304147+0000","last_unstale":"2024-09-12T03:26:14.669870+0000","last_undegraded":"2024-09-12T03:26:14.669870+0000","last_fullsized":"2024-09-12T03:26:14.669870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:17.834282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660722+0000","last_change":"2024-09-12T03:24:40.706844+0000","last_active":"2024-09-12T03:26:14.660722+0000","last_peered":"2024-09-12T03:26:14.660722+0000","last_clean":"2024-09-12T03:26:14.660722+0000","last_became_active":"2024-09-12T03:24:40.704829+0000","last_became_peered":"2024-09-12T03:24:40.704829+0000","last_unstale":"2024-09-12T03:26:14.660722+0000","last_undegraded":"2024-09-12T03:26:14.660722+0000","last_fullsized":"2024-09-12T03:26:14.660722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:32.875316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684487+0000","last_change":"2024-09-12T03:24:31.650168+0000","last_active":"2024-09-12T03:26:14.684487+0000","last_peered":"2024-09-12T03:26:14.684487+0000","last_clean":"2024-09-12T03:26:14.684487+0000","last_became_active":"2024-09-12T03:24:31.649754+0000","last_became_peered":"2024-09-12T03:24:31.649754+0000","last_unstale":"2024-09-12T03:26:14.684487+0000","last_undegraded":"2024-09-12T03:26:14.684487+0000","last_fullsized":"2024-09-12T03:26:14.684487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:09.615836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670514+0000","last_change":"2024-09-12T03:24:29.303488+0000","last_active":"2024-09-12T03:26:14.670514+0000","last_peered":"2024-09-12T03:26:14.670514+0000","last_clean":"2024-09-12T03:26:14.670514+0000","last_became_active":"2024-09-12T03:24:29.301858+0000","last_became_peered":"2024-09-12T03:24:29.301858+0000","last_unstale":"2024-09-12T03:26:14.670514+0000","last_undegraded":"2024-09-12T03:26:14.670514+0000","last_fullsized":"2024-09-12T03:26:14.670514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:27.846820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667760+0000","last_change":"2024-09-12T03:24:40.697734+0000","last_active":"2024-09-12T03:26:14.667760+0000","last_peered":"2024-09-12T03:26:14.667760+0000","last_clean":"2024-09-12T03:26:14.667760+0000","last_became_active":"2024-09-12T03:24:40.697271+0000","last_became_peered":"2024-09-12T03:24:40.697271+0000","last_unstale":"2024-09-12T03:26:14.667760+0000","last_undegraded":"2024-09-12T03:26:14.667760+0000","last_fullsized":"2024-09-12T03:26:14.667760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.336322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684740+0000","last_change":"2024-09-12T03:24:31.647040+0000","last_active":"2024-09-12T03:26:14.684740+0000","last_peered":"2024-09-12T03:26:14.684740+0000","last_clean":"2024-09-12T03:26:14.684740+0000","last_became_active":"2024-09-12T03:24:31.643365+0000","last_became_peered":"2024-09-12T03:24:31.643365+0000","last_unstale":"2024-09-12T03:26:14.684740+0000","last_undegraded":"2024-09-12T03:26:14.684740+0000","last_fullsized":"2024-09-12T03:26:14.684740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:06.638005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663708+0000","last_change":"2024-09-12T03:24:29.303390+0000","last_active":"2024-09-12T03:26:14.663708+0000","last_peered":"2024-09-12T03:26:14.663708+0000","last_clean":"2024-09-12T03:26:14.663708+0000","last_became_active":"2024-09-12T03:24:29.303072+0000","last_became_peered":"2024-09-12T03:24:29.303072+0000","last_unstale":"2024-09-12T03:26:14.663708+0000","last_undegraded":"2024-09-12T03:26:14.663708+0000","last_fullsized":"2024-09-12T03:26:14.663708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:01.572888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670400+0000","last_change":"2024-09-12T03:24:40.707977+0000","last_active":"2024-09-12T03:26:14.670400+0000","last_peered":"2024-09-12T03:26:14.670400+0000","last_clean":"2024-09-12T03:26:14.670400+0000","last_became_active":"2024-09-12T03:24:40.707497+0000","last_became_peered":"2024-09-12T03:24:40.707497+0000","last_unstale":"2024-09-12T03:26:14.670400+0000","last_undegraded":"2024-09-12T03:26:14.670400+0000","last_fullsized":"2024-09-12T03:26:14.670400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:45.474021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665523+0000","last_change":"2024-09-12T03:24:31.642417+0000","last_active":"2024-09-12T03:26:14.665523+0000","last_peered":"2024-09-12T03:26:14.665523+0000","last_clean":"2024-09-12T03:26:14.665523+0000","last_became_active":"2024-09-12T03:24:31.642076+0000","last_became_peered":"2024-09-12T03:24:31.642076+0000","last_unstale":"2024-09-12T03:26:14.665523+0000","last_undegraded":"2024-09-12T03:26:14.665523+0000","last_fullsized":"2024-09-12T03:26:14.665523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:56.436088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:18.954263+0000","last_change":"2024-09-12T03:24:29.302982+0000","last_active":"2024-09-12T03:26:18.954263+0000","last_peered":"2024-09-12T03:26:18.954263+0000","last_clean":"2024-09-12T03:26:18.954263+0000","last_became_active":"2024-09-12T03:24:29.302369+0000","last_became_peered":"2024-09-12T03:24:29.302369+0000","last_unstale":"2024-09-12T03:26:18.954263+0000","last_undegraded":"2024-09-12T03:26:18.954263+0000","last_fullsized":"2024-09-12T03:26:18.954263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:24:30.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":124,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660148+0000","last_change":"2024-09-12T03:24:40.696745+0000","last_active":"2024-09-12T03:26:14.660148+0000","last_peered":"2024-09-12T03:26:14.660148+0000","last_clean":"2024-09-12T03:26:14.660148+0000","last_became_active":"2024-09-12T03:24:40.696273+0000","last_became_peered":"2024-09-12T03:24:40.696273+0000","last_unstale":"2024-09-12T03:26:14.660148+0000","last_undegraded":"2024-09-12T03:26:14.660148+0000","last_fullsized":"2024-09-12T03:26:14.660148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.519765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670523+0000","last_change":"2024-09-12T03:24:31.650567+0000","last_active":"2024-09-12T03:26:14.670523+0000","last_peered":"2024-09-12T03:26:14.670523+0000","last_clean":"2024-09-12T03:26:14.670523+0000","last_became_active":"2024-09-12T03:24:31.650242+0000","last_became_peered":"2024-09-12T03:24:31.650242+0000","last_unstale":"2024-09-12T03:26:14.670523+0000","last_undegraded":"2024-09-12T03:26:14.670523+0000","last_fullsized":"2024-09-12T03:26:14.670523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:14.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670630+0000","last_change":"2024-09-12T03:24:29.310630+0000","last_active":"2024-09-12T03:26:14.670630+0000","last_peered":"2024-09-12T03:26:14.670630+0000","last_clean":"2024-09-12T03:26:14.670630+0000","last_became_active":"2024-09-12T03:24:29.310302+0000","last_became_peered":"2024-09-12T03:24:29.310302+0000","last_unstale":"2024-09-12T03:26:14.670630+0000","last_undegraded":"2024-09-12T03:26:14.670630+0000","last_fullsized":"2024-09-12T03:26:14.670630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:53.835246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669376+0000","last_change":"2024-09-12T03:24:40.698941+0000","last_active":"2024-09-12T03:26:14.669376+0000","last_peered":"2024-09-12T03:26:14.669376+0000","last_clean":"2024-09-12T03:26:14.669376+0000","last_became_active":"2024-09-12T03:24:40.698323+0000","last_became_peered":"2024-09-12T03:24:40.698323+0000","last_unstale":"2024-09-12T03:26:14.669376+0000","last_undegraded":"2024-09-12T03:26:14.669376+0000","last_fullsized":"2024-09-12T03:26:14.669376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:29.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669098+0000","last_change":"2024-09-12T03:25:14.320685+0000","last_active":"2024-09-12T03:26:14.669098+0000","last_peered":"2024-09-12T03:26:14.669098+0000","last_clean":"2024-09-12T03:26:14.669098+0000","last_became_active":"2024-09-12T03:25:14.320295+0000","last_became_peered":"2024-09-12T03:25:14.320295+0000","last_unstale":"2024-09-12T03:26:14.669098+0000","last_undegraded":"2024-09-12T03:26:14.669098+0000","last_fullsized":"2024-09-12T03:26:14.669098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:42.459136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684539+0000","last_change":"2024-09-12T03:24:29.302988+0000","last_active":"2024-09-12T03:26:14.684539+0000","last_peered":"2024-09-12T03:26:14.684539+0000","last_clean":"2024-09-12T03:26:14.684539+0000","last_became_active":"2024-09-12T03:24:29.302569+0000","last_became_peered":"2024-09-12T03:24:29.302569+0000","last_unstale":"2024-09-12T03:26:14.684539+0000","last_undegraded":"2024-09-12T03:26:14.684539+0000","last_fullsized":"2024-09-12T03:26:14.684539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:00.196284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660084+0000","last_change":"2024-09-12T03:24:40.701486+0000","last_active":"2024-09-12T03:26:14.660084+0000","last_peered":"2024-09-12T03:26:14.660084+0000","last_clean":"2024-09-12T03:26:14.660084+0000","last_became_active":"2024-09-12T03:24:40.701078+0000","last_became_peered":"2024-09-12T03:24:40.701078+0000","last_unstale":"2024-09-12T03:26:14.660084+0000","last_undegraded":"2024-09-12T03:26:14.660084+0000","last_fullsized":"2024-09-12T03:26:14.660084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:50.032773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670112+0000","last_change":"2024-09-12T03:24:31.646681+0000","last_active":"2024-09-12T03:26:14.670112+0000","last_peered":"2024-09-12T03:26:14.670112+0000","last_clean":"2024-09-12T03:26:14.670112+0000","last_became_active":"2024-09-12T03:24:31.646389+0000","last_became_peered":"2024-09-12T03:24:31.646389+0000","last_unstale":"2024-09-12T03:26:14.670112+0000","last_undegraded":"2024-09-12T03:26:14.670112+0000","last_fullsized":"2024-09-12T03:26:14.670112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:37.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":98,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:17.946602+0000","last_change":"2024-09-12T03:24:29.299316+0000","last_active":"2024-09-12T03:26:17.946602+0000","last_peered":"2024-09-12T03:26:17.946602+0000","last_clean":"2024-09-12T03:26:17.946602+0000","last_became_active":"2024-09-12T03:24:29.298900+0000","last_became_peered":"2024-09-12T03:24:29.298900+0000","last_unstale":"2024-09-12T03:26:17.946602+0000","last_undegraded":"2024-09-12T03:26:17.946602+0000","last_fullsized":"2024-09-12T03:26:17.946602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:36.650938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684874+0000","last_change":"2024-09-12T03:24:40.702034+0000","last_active":"2024-09-12T03:26:14.684874+0000","last_peered":"2024-09-12T03:26:14.684874+0000","last_clean":"2024-09-12T03:26:14.684874+0000","last_became_active":"2024-09-12T03:24:40.699312+0000","last_became_peered":"2024-09-12T03:24:40.699312+0000","last_unstale":"2024-09-12T03:26:14.684874+0000","last_undegraded":"2024-09-12T03:26:14.684874+0000","last_fullsized":"2024-09-12T03:26:14.684874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:58.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668712+0000","last_change":"2024-09-12T03:24:40.701744+0000","last_active":"2024-09-12T03:26:14.668712+0000","last_peered":"2024-09-12T03:26:14.668712+0000","last_clean":"2024-09-12T03:26:14.668712+0000","last_became_active":"2024-09-12T03:24:40.700929+0000","last_became_peered":"2024-09-12T03:24:40.700929+0000","last_unstale":"2024-09-12T03:26:14.668712+0000","last_undegraded":"2024-09-12T03:26:14.668712+0000","last_fullsized":"2024-09-12T03:26:14.668712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.851810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660177+0000","last_change":"2024-09-12T03:24:31.647774+0000","last_active":"2024-09-12T03:26:14.660177+0000","last_peered":"2024-09-12T03:26:14.660177+0000","last_clean":"2024-09-12T03:26:14.660177+0000","last_became_active":"2024-09-12T03:24:31.647329+0000","last_became_peered":"2024-09-12T03:24:31.647329+0000","last_unstale":"2024-09-12T03:26:14.660177+0000","last_undegraded":"2024-09-12T03:26:14.660177+0000","last_fullsized":"2024-09-12T03:26:14.660177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:00.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670540+0000","last_change":"2024-09-12T03:24:29.307455+0000","last_active":"2024-09-12T03:26:14.670540+0000","last_peered":"2024-09-12T03:26:14.670540+0000","last_clean":"2024-09-12T03:26:14.670540+0000","last_became_active":"2024-09-12T03:24:29.307120+0000","last_became_peered":"2024-09-12T03:24:29.307120+0000","last_unstale":"2024-09-12T03:26:14.670540+0000","last_undegraded":"2024-09-12T03:26:14.670540+0000","last_fullsized":"2024-09-12T03:26:14.670540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:29.374097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667548+0000","last_change":"2024-09-12T03:24:40.695149+0000","last_active":"2024-09-12T03:26:14.667548+0000","last_peered":"2024-09-12T03:26:14.667548+0000","last_clean":"2024-09-12T03:26:14.667548+0000","last_became_active":"2024-09-12T03:24:40.694755+0000","last_became_peered":"2024-09-12T03:24:40.694755+0000","last_unstale":"2024-09-12T03:26:14.667548+0000","last_undegraded":"2024-09-12T03:26:14.667548+0000","last_fullsized":"2024-09-12T03:26:14.667548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:34.045481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684978+0000","last_change":"2024-09-12T03:24:31.654674+0000","last_active":"2024-09-12T03:26:14.684978+0000","last_peered":"2024-09-12T03:26:14.684978+0000","last_clean":"2024-09-12T03:26:14.684978+0000","last_became_active":"2024-09-12T03:24:31.654319+0000","last_became_peered":"2024-09-12T03:24:31.654319+0000","last_unstale":"2024-09-12T03:26:14.684978+0000","last_undegraded":"2024-09-12T03:26:14.684978+0000","last_fullsized":"2024-09-12T03:26:14.684978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:33.441162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684538+0000","last_change":"2024-09-12T03:24:29.307849+0000","last_active":"2024-09-12T03:26:14.684538+0000","last_peered":"2024-09-12T03:26:14.684538+0000","last_clean":"2024-09-12T03:26:14.684538+0000","last_became_active":"2024-09-12T03:24:29.306017+0000","last_became_peered":"2024-09-12T03:24:29.306017+0000","last_unstale":"2024-09-12T03:26:14.684538+0000","last_undegraded":"2024-09-12T03:26:14.684538+0000","last_fullsized":"2024-09-12T03:26:14.684538+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:37.478289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663950+0000","last_change":"2024-09-12T03:24:29.302410+0000","last_active":"2024-09-12T03:26:14.663950+0000","last_peered":"2024-09-12T03:26:14.663950+0000","last_clean":"2024-09-12T03:26:14.663950+0000","last_became_active":"2024-09-12T03:24:29.302031+0000","last_became_peered":"2024-09-12T03:24:29.302031+0000","last_unstale":"2024-09-12T03:26:14.663950+0000","last_undegraded":"2024-09-12T03:26:14.663950+0000","last_fullsized":"2024-09-12T03:26:14.663950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:11.698690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670272+0000","last_change":"2024-09-12T03:24:31.652823+0000","last_active":"2024-09-12T03:26:14.670272+0000","last_peered":"2024-09-12T03:26:14.670272+0000","last_clean":"2024-09-12T03:26:14.670272+0000","last_became_active":"2024-09-12T03:24:31.652438+0000","last_became_peered":"2024-09-12T03:24:31.652438+0000","last_unstale":"2024-09-12T03:26:14.670272+0000","last_undegraded":"2024-09-12T03:26:14.670272+0000","last_fullsized":"2024-09-12T03:26:14.670272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663716+0000","last_change":"2024-09-12T03:25:13.942339+0000","last_active":"2024-09-12T03:26:14.663716+0000","last_peered":"2024-09-12T03:26:14.663716+0000","last_clean":"2024-09-12T03:26:14.663716+0000","last_became_active":"2024-09-12T03:25:13.942030+0000","last_became_peered":"2024-09-12T03:25:13.942030+0000","last_unstale":"2024-09-12T03:26:14.663716+0000","last_undegraded":"2024-09-12T03:26:14.663716+0000","last_fullsized":"2024-09-12T03:26:14.663716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:39.584996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:17.947215+0000","last_change":"2024-09-12T03:24:29.303318+0000","last_active":"2024-09-12T03:26:17.947215+0000","last_peered":"2024-09-12T03:26:17.947215+0000","last_clean":"2024-09-12T03:26:17.947215+0000","last_became_active":"2024-09-12T03:24:29.303001+0000","last_became_peered":"2024-09-12T03:24:29.303001+0000","last_unstale":"2024-09-12T03:26:17.947215+0000","last_undegraded":"2024-09-12T03:26:17.947215+0000","last_fullsized":"2024-09-12T03:26:17.947215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:08.094697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665180+0000","last_change":"2024-09-12T03:24:31.644465+0000","last_active":"2024-09-12T03:26:14.665180+0000","last_peered":"2024-09-12T03:26:14.665180+0000","last_clean":"2024-09-12T03:26:14.665180+0000","last_became_active":"2024-09-12T03:24:31.644141+0000","last_became_peered":"2024-09-12T03:24:31.644141+0000","last_unstale":"2024-09-12T03:26:14.665180+0000","last_undegraded":"2024-09-12T03:26:14.665180+0000","last_fullsized":"2024-09-12T03:26:14.665180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:37.448614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663896+0000","last_change":"2024-09-12T03:24:40.695190+0000","last_active":"2024-09-12T03:26:14.663896+0000","last_peered":"2024-09-12T03:26:14.663896+0000","last_clean":"2024-09-12T03:26:14.663896+0000","last_became_active":"2024-09-12T03:24:40.694746+0000","last_became_peered":"2024-09-12T03:24:40.694746+0000","last_unstale":"2024-09-12T03:26:14.663896+0000","last_undegraded":"2024-09-12T03:26:14.663896+0000","last_fullsized":"2024-09-12T03:26:14.663896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:31.905437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667099+0000","last_change":"2024-09-12T03:24:31.649068+0000","last_active":"2024-09-12T03:26:14.667099+0000","last_peered":"2024-09-12T03:26:14.667099+0000","last_clean":"2024-09-12T03:26:14.667099+0000","last_became_active":"2024-09-12T03:24:31.648761+0000","last_became_peered":"2024-09-12T03:24:31.648761+0000","last_unstale":"2024-09-12T03:26:14.667099+0000","last_undegraded":"2024-09-12T03:26:14.667099+0000","last_fullsized":"2024-09-12T03:26:14.667099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:08.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663592+0000","last_change":"2024-09-12T03:24:31.643604+0000","last_active":"2024-09-12T03:26:14.663592+0000","last_peered":"2024-09-12T03:26:14.663592+0000","last_clean":"2024-09-12T03:26:14.663592+0000","last_became_active":"2024-09-12T03:24:31.643053+0000","last_became_peered":"2024-09-12T03:24:31.643053+0000","last_unstale":"2024-09-12T03:26:14.663592+0000","last_undegraded":"2024-09-12T03:26:14.663592+0000","last_fullsized":"2024-09-12T03:26:14.663592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:59.814891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663349+0000","last_change":"2024-09-12T03:24:31.648376+0000","last_active":"2024-09-12T03:26:14.663349+0000","last_peered":"2024-09-12T03:26:14.663349+0000","last_clean":"2024-09-12T03:26:14.663349+0000","last_became_active":"2024-09-12T03:24:31.648076+0000","last_became_peered":"2024-09-12T03:24:31.648076+0000","last_unstale":"2024-09-12T03:26:14.663349+0000","last_undegraded":"2024-09-12T03:26:14.663349+0000","last_fullsized":"2024-09-12T03:26:14.663349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:47.736362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668249+0000","last_change":"2024-09-12T03:24:31.652489+0000","last_active":"2024-09-12T03:26:14.668249+0000","last_peered":"2024-09-12T03:26:14.668249+0000","last_clean":"2024-09-12T03:26:14.668249+0000","last_became_active":"2024-09-12T03:24:31.651907+0000","last_became_peered":"2024-09-12T03:24:31.651907+0000","last_unstale":"2024-09-12T03:26:14.668249+0000","last_undegraded":"2024-09-12T03:26:14.668249+0000","last_fullsized":"2024-09-12T03:26:14.668249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:24.584264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684936+0000","last_change":"2024-09-12T03:24:31.649967+0000","last_active":"2024-09-12T03:26:14.684936+0000","last_peered":"2024-09-12T03:26:14.684936+0000","last_clean":"2024-09-12T03:26:14.684936+0000","last_became_active":"2024-09-12T03:24:31.649687+0000","last_became_peered":"2024-09-12T03:24:31.649687+0000","last_unstale":"2024-09-12T03:26:14.684936+0000","last_undegraded":"2024-09-12T03:26:14.684936+0000","last_fullsized":"2024-09-12T03:26:14.684936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:14.875352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667562+0000","last_change":"2024-09-12T03:24:31.645115+0000","last_active":"2024-09-12T03:26:14.667562+0000","last_peered":"2024-09-12T03:26:14.667562+0000","last_clean":"2024-09-12T03:26:14.667562+0000","last_became_active":"2024-09-12T03:24:31.644442+0000","last_became_peered":"2024-09-12T03:24:31.644442+0000","last_unstale":"2024-09-12T03:26:14.667562+0000","last_undegraded":"2024-09-12T03:26:14.667562+0000","last_fullsized":"2024-09-12T03:26:14.667562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:28.539948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.664091+0000","last_change":"2024-09-12T03:24:31.643283+0000","last_active":"2024-09-12T03:26:14.664091+0000","last_peered":"2024-09-12T03:26:14.664091+0000","last_clean":"2024-09-12T03:26:14.664091+0000","last_became_active":"2024-09-12T03:24:31.642935+0000","last_became_peered":"2024-09-12T03:24:31.642935+0000","last_unstale":"2024-09-12T03:26:14.664091+0000","last_undegraded":"2024-09-12T03:26:14.664091+0000","last_fullsized":"2024-09-12T03:26:14.664091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:30.754875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660623+0000","last_change":"2024-09-12T03:25:13.817610+0000","last_active":"2024-09-12T03:26:14.660623+0000","last_peered":"2024-09-12T03:26:14.660623+0000","last_clean":"2024-09-12T03:26:14.660623+0000","last_became_active":"2024-09-12T03:25:13.817008+0000","last_became_peered":"2024-09-12T03:25:13.817008+0000","last_unstale":"2024-09-12T03:26:14.660623+0000","last_undegraded":"2024-09-12T03:26:14.660623+0000","last_fullsized":"2024-09-12T03:26:14.660623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:45.334353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670205+0000","last_change":"2024-09-12T03:24:31.656588+0000","last_active":"2024-09-12T03:26:14.670205+0000","last_peered":"2024-09-12T03:26:14.670205+0000","last_clean":"2024-09-12T03:26:14.670205+0000","last_became_active":"2024-09-12T03:24:31.656294+0000","last_became_peered":"2024-09-12T03:24:31.656294+0000","last_unstale":"2024-09-12T03:26:14.670205+0000","last_undegraded":"2024-09-12T03:26:14.670205+0000","last_fullsized":"2024-09-12T03:26:14.670205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.855802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685017+0000","last_change":"2024-09-12T03:24:31.648986+0000","last_active":"2024-09-12T03:26:14.685017+0000","last_peered":"2024-09-12T03:26:14.685017+0000","last_clean":"2024-09-12T03:26:14.685017+0000","last_became_active":"2024-09-12T03:24:31.648144+0000","last_became_peered":"2024-09-12T03:24:31.648144+0000","last_unstale":"2024-09-12T03:26:14.685017+0000","last_undegraded":"2024-09-12T03:26:14.685017+0000","last_fullsized":"2024-09-12T03:26:14.685017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.823742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670391+0000","last_change":"2024-09-12T03:24:31.644390+0000","last_active":"2024-09-12T03:26:14.670391+0000","last_peered":"2024-09-12T03:26:14.670391+0000","last_clean":"2024-09-12T03:26:14.670391+0000","last_became_active":"2024-09-12T03:24:31.642660+0000","last_became_peered":"2024-09-12T03:24:31.642660+0000","last_unstale":"2024-09-12T03:26:14.670391+0000","last_undegraded":"2024-09-12T03:26:14.670391+0000","last_fullsized":"2024-09-12T03:26:14.670391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:58.577883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685264+0000","last_change":"2024-09-12T03:24:31.649895+0000","last_active":"2024-09-12T03:26:14.685264+0000","last_peered":"2024-09-12T03:26:14.685264+0000","last_clean":"2024-09-12T03:26:14.685264+0000","last_became_active":"2024-09-12T03:24:31.649141+0000","last_became_peered":"2024-09-12T03:24:31.649141+0000","last_unstale":"2024-09-12T03:26:14.685264+0000","last_undegraded":"2024-09-12T03:26:14.685264+0000","last_fullsized":"2024-09-12T03:26:14.685264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:48.175496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667879+0000","last_change":"2024-09-12T03:24:31.640402+0000","last_active":"2024-09-12T03:26:14.667879+0000","last_peered":"2024-09-12T03:26:14.667879+0000","last_clean":"2024-09-12T03:26:14.667879+0000","last_became_active":"2024-09-12T03:24:31.639987+0000","last_became_peered":"2024-09-12T03:24:31.639987+0000","last_unstale":"2024-09-12T03:26:14.667879+0000","last_undegraded":"2024-09-12T03:26:14.667879+0000","last_fullsized":"2024-09-12T03:26:14.667879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:51.540149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.664073+0000","last_change":"2024-09-12T03:24:31.647524+0000","last_active":"2024-09-12T03:26:14.664073+0000","last_peered":"2024-09-12T03:26:14.664073+0000","last_clean":"2024-09-12T03:26:14.664073+0000","last_became_active":"2024-09-12T03:24:31.647204+0000","last_became_peered":"2024-09-12T03:24:31.647204+0000","last_unstale":"2024-09-12T03:26:14.664073+0000","last_undegraded":"2024-09-12T03:26:14.664073+0000","last_fullsized":"2024-09-12T03:26:14.664073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.895586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660706+0000","last_change":"2024-09-12T03:24:31.640797+0000","last_active":"2024-09-12T03:26:14.660706+0000","last_peered":"2024-09-12T03:26:14.660706+0000","last_clean":"2024-09-12T03:26:14.660706+0000","last_became_active":"2024-09-12T03:24:31.640479+0000","last_became_peered":"2024-09-12T03:24:31.640479+0000","last_unstale":"2024-09-12T03:26:14.660706+0000","last_undegraded":"2024-09-12T03:26:14.660706+0000","last_fullsized":"2024-09-12T03:26:14.660706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:08.762939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667947+0000","last_change":"2024-09-12T03:24:31.644225+0000","last_active":"2024-09-12T03:26:14.667947+0000","last_peered":"2024-09-12T03:26:14.667947+0000","last_clean":"2024-09-12T03:26:14.667947+0000","last_became_active":"2024-09-12T03:24:31.643877+0000","last_became_peered":"2024-09-12T03:24:31.643877+0000","last_unstale":"2024-09-12T03:26:14.667947+0000","last_undegraded":"2024-09-12T03:26:14.667947+0000","last_fullsized":"2024-09-12T03:26:14.667947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:50.511392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669839+0000","last_change":"2024-09-12T03:24:31.644917+0000","last_active":"2024-09-12T03:26:14.669839+0000","last_peered":"2024-09-12T03:26:14.669839+0000","last_clean":"2024-09-12T03:26:14.669839+0000","last_became_active":"2024-09-12T03:24:31.644591+0000","last_became_peered":"2024-09-12T03:24:31.644591+0000","last_unstale":"2024-09-12T03:26:14.669839+0000","last_undegraded":"2024-09-12T03:26:14.669839+0000","last_fullsized":"2024-09-12T03:26:14.669839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:21.961959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668067+0000","last_change":"2024-09-12T03:24:31.648146+0000","last_active":"2024-09-12T03:26:14.668067+0000","last_peered":"2024-09-12T03:26:14.668067+0000","last_clean":"2024-09-12T03:26:14.668067+0000","last_became_active":"2024-09-12T03:24:31.641423+0000","last_became_peered":"2024-09-12T03:24:31.641423+0000","last_unstale":"2024-09-12T03:26:14.668067+0000","last_undegraded":"2024-09-12T03:26:14.668067+0000","last_fullsized":"2024-09-12T03:26:14.668067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:50.985126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668440+0000","last_change":"2024-09-12T03:24:31.641030+0000","last_active":"2024-09-12T03:26:14.668440+0000","last_peered":"2024-09-12T03:26:14.668440+0000","last_clean":"2024-09-12T03:26:14.668440+0000","last_became_active":"2024-09-12T03:24:31.639015+0000","last_became_peered":"2024-09-12T03:24:31.639015+0000","last_unstale":"2024-09-12T03:26:14.668440+0000","last_undegraded":"2024-09-12T03:26:14.668440+0000","last_fullsized":"2024-09-12T03:26:14.668440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.961304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685232+0000","last_change":"2024-09-12T03:24:31.644601+0000","last_active":"2024-09-12T03:26:14.685232+0000","last_peered":"2024-09-12T03:26:14.685232+0000","last_clean":"2024-09-12T03:26:14.685232+0000","last_became_active":"2024-09-12T03:24:31.644122+0000","last_became_peered":"2024-09-12T03:24:31.644122+0000","last_unstale":"2024-09-12T03:26:14.685232+0000","last_undegraded":"2024-09-12T03:26:14.685232+0000","last_fullsized":"2024-09-12T03:26:14.685232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:06.335464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670199+0000","last_change":"2024-09-12T03:24:31.641496+0000","last_active":"2024-09-12T03:26:14.670199+0000","last_peered":"2024-09-12T03:26:14.670199+0000","last_clean":"2024-09-12T03:26:14.670199+0000","last_became_active":"2024-09-12T03:24:31.641195+0000","last_became_peered":"2024-09-12T03:24:31.641195+0000","last_unstale":"2024-09-12T03:26:14.670199+0000","last_undegraded":"2024-09-12T03:26:14.670199+0000","last_fullsized":"2024-09-12T03:26:14.670199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:02.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684560+0000","last_change":"2024-09-12T03:24:31.645766+0000","last_active":"2024-09-12T03:26:14.684560+0000","last_peered":"2024-09-12T03:26:14.684560+0000","last_clean":"2024-09-12T03:26:14.684560+0000","last_became_active":"2024-09-12T03:24:31.644903+0000","last_became_peered":"2024-09-12T03:24:31.644903+0000","last_unstale":"2024-09-12T03:26:14.684560+0000","last_undegraded":"2024-09-12T03:26:14.684560+0000","last_fullsized":"2024-09-12T03:26:14.684560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.664128+0000","last_change":"2024-09-12T03:25:13.813333+0000","last_active":"2024-09-12T03:26:14.664128+0000","last_peered":"2024-09-12T03:26:14.664128+0000","last_clean":"2024-09-12T03:26:14.664128+0000","last_became_active":"2024-09-12T03:25:13.812963+0000","last_became_peered":"2024-09-12T03:25:13.812963+0000","last_unstale":"2024-09-12T03:26:14.664128+0000","last_undegraded":"2024-09-12T03:26:14.664128+0000","last_fullsized":"2024-09-12T03:26:14.664128+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:38.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670223+0000","last_change":"2024-09-12T03:25:14.266892+0000","last_active":"2024-09-12T03:26:14.670223+0000","last_peered":"2024-09-12T03:26:14.670223+0000","last_clean":"2024-09-12T03:26:14.670223+0000","last_became_active":"2024-09-12T03:25:14.266452+0000","last_became_peered":"2024-09-12T03:25:14.266452+0000","last_unstale":"2024-09-12T03:26:14.670223+0000","last_undegraded":"2024-09-12T03:26:14.670223+0000","last_fullsized":"2024-09-12T03:26:14.670223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:58.206639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684877+0000","last_change":"2024-09-12T03:24:31.646199+0000","last_active":"2024-09-12T03:26:14.684877+0000","last_peered":"2024-09-12T03:26:14.684877+0000","last_clean":"2024-09-12T03:26:14.684877+0000","last_became_active":"2024-09-12T03:24:31.645872+0000","last_became_peered":"2024-09-12T03:24:31.645872+0000","last_unstale":"2024-09-12T03:26:14.684877+0000","last_undegraded":"2024-09-12T03:26:14.684877+0000","last_fullsized":"2024-09-12T03:26:14.684877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:21.538730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684198+0000","last_change":"2024-09-12T03:24:31.643704+0000","last_active":"2024-09-12T03:26:14.684198+0000","last_peered":"2024-09-12T03:26:14.684198+0000","last_clean":"2024-09-12T03:26:14.684198+0000","last_became_active":"2024-09-12T03:24:31.643190+0000","last_became_peered":"2024-09-12T03:24:31.643190+0000","last_unstale":"2024-09-12T03:26:14.684198+0000","last_undegraded":"2024-09-12T03:26:14.684198+0000","last_fullsized":"2024-09-12T03:26:14.684198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:23.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663836+0000","last_change":"2024-09-12T03:24:31.643282+0000","last_active":"2024-09-12T03:26:14.663836+0000","last_peered":"2024-09-12T03:26:14.663836+0000","last_clean":"2024-09-12T03:26:14.663836+0000","last_became_active":"2024-09-12T03:24:31.642721+0000","last_became_peered":"2024-09-12T03:24:31.642721+0000","last_unstale":"2024-09-12T03:26:14.663836+0000","last_undegraded":"2024-09-12T03:26:14.663836+0000","last_fullsized":"2024-09-12T03:26:14.663836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:52.025426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665179+0000","last_change":"2024-09-12T03:24:31.650266+0000","last_active":"2024-09-12T03:26:14.665179+0000","last_peered":"2024-09-12T03:26:14.665179+0000","last_clean":"2024-09-12T03:26:14.665179+0000","last_became_active":"2024-09-12T03:24:31.649887+0000","last_became_peered":"2024-09-12T03:24:31.649887+0000","last_unstale":"2024-09-12T03:26:14.665179+0000","last_undegraded":"2024-09-12T03:26:14.665179+0000","last_fullsized":"2024-09-12T03:26:14.665179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:05.043430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669867+0000","last_change":"2024-09-12T03:24:31.639381+0000","last_active":"2024-09-12T03:26:14.669867+0000","last_peered":"2024-09-12T03:26:14.669867+0000","last_clean":"2024-09-12T03:26:14.669867+0000","last_became_active":"2024-09-12T03:24:31.639047+0000","last_became_peered":"2024-09-12T03:24:31.639047+0000","last_unstale":"2024-09-12T03:26:14.669867+0000","last_undegraded":"2024-09-12T03:26:14.669867+0000","last_fullsized":"2024-09-12T03:26:14.669867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:57.608026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668170+0000","last_change":"2024-09-12T03:24:31.646682+0000","last_active":"2024-09-12T03:26:14.668170+0000","last_peered":"2024-09-12T03:26:14.668170+0000","last_clean":"2024-09-12T03:26:14.668170+0000","last_became_active":"2024-09-12T03:24:31.645065+0000","last_became_peered":"2024-09-12T03:24:31.645065+0000","last_unstale":"2024-09-12T03:26:14.668170+0000","last_undegraded":"2024-09-12T03:26:14.668170+0000","last_fullsized":"2024-09-12T03:26:14.668170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:27.328188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668654+0000","last_change":"2024-09-12T03:24:31.641532+0000","last_active":"2024-09-12T03:26:14.668654+0000","last_peered":"2024-09-12T03:26:14.668654+0000","last_clean":"2024-09-12T03:26:14.668654+0000","last_became_active":"2024-09-12T03:24:31.641164+0000","last_became_peered":"2024-09-12T03:24:31.641164+0000","last_unstale":"2024-09-12T03:26:14.668654+0000","last_undegraded":"2024-09-12T03:26:14.668654+0000","last_fullsized":"2024-09-12T03:26:14.668654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:28.984409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667931+0000","last_change":"2024-09-12T03:24:31.646372+0000","last_active":"2024-09-12T03:26:14.667931+0000","last_peered":"2024-09-12T03:26:14.667931+0000","last_clean":"2024-09-12T03:26:14.667931+0000","last_became_active":"2024-09-12T03:24:31.644636+0000","last_became_peered":"2024-09-12T03:24:31.644636+0000","last_unstale":"2024-09-12T03:26:14.667931+0000","last_undegraded":"2024-09-12T03:26:14.667931+0000","last_fullsized":"2024-09-12T03:26:14.667931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:51.023245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":363,"num_read_kb":254,"num_write":169,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481827,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1081344,"data_stored":538749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1769472,"data_stored":1190059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906989,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":543059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1089536,"data_stored":538861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1687552,"data_stored":1134794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1073152,"data_stored":534139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1695744,"data_stored":1130382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1089536,"data_stored":539371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":534333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1155072,"data_stored":599162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1150976,"data_stored":598225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":535527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:26:22.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:26:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:22 smithi142 ceph-mon[35329]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 16 B/s, 1 objects/s recovering 2024-09-12T03:26:23.094 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:23.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:22 smithi082 ceph-mon[35046]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 16 B/s, 1 objects/s recovering 2024-09-12T03:26:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:22 smithi050 ceph-mon[30378]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 16 B/s, 1 objects/s recovering 2024-09-12T03:26:23.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 -- 172.21.15.50:0/3697041083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 msgr2=0x7f1d681043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:23.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 --2- 172.21.15.50:0/3697041083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d681043d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009900 tx=0x7f1d5c02a400 comp rx=0 tx=0).stop 2024-09-12T03:26:23.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 -- 172.21.15.50:0/3697041083 shutdown_connections 2024-09-12T03:26:23.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 --2- 172.21.15.50:0/3697041083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68109120 0x7f1d68110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 --2- 172.21.15.50:0/3697041083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d681049a0 0x7f1d681089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 --2- 172.21.15.50:0/3697041083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d681043d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 -- 172.21.15.50:0/3697041083 >> 172.21.15.50:0/3697041083 conn(0x7f1d680ffa90 msgr2=0x7f1d68101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:23.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 -- 172.21.15.50:0/3697041083 shutdown_connections 2024-09-12T03:26:23.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.643+0000 7f1d70800640 1 -- 172.21.15.50:0/3697041083 wait complete. 2024-09-12T03:26:23.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.644+0000 7f1d70800640 1 Processor -- start 2024-09-12T03:26:23.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.644+0000 7f1d70800640 1 -- start start 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d70800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d6819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d70800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d681049a0 0x7f1d6819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d70800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d68109120 0x7f1d6819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d70800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68114140 con 0x7f1d68103ff0 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d70800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68113fc0 con 0x7f1d681049a0 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d70800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681142c0 con 0x7f1d68109120 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d6de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d6819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:23.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d6de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d6819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49782/0 (socket says 172.21.15.50:49782) 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d6de00640 1 -- 172.21.15.50:0/1135513344 learned_addr learned my addr 172.21.15.50:0/1135513344 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.645+0000 7f1d6d400640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d681049a0 0x7f1d6819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6e800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d68109120 0x7f1d6819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6de00640 1 -- 172.21.15.50:0/1135513344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d68109120 msgr2=0x7f1d6819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6de00640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d68109120 0x7f1d6819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6de00640 1 -- 172.21.15.50:0/1135513344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d681049a0 msgr2=0x7f1d6819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:23.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6de00640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d681049a0 0x7f1d6819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6de00640 1 -- 172.21.15.50:0/1135513344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6819f5b0 con 0x7f1d68103ff0 2024-09-12T03:26:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.646+0000 7f1d6e800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d68109120 0x7f1d6819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.647+0000 7f1d6de00640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d6819a600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c006010 tx=0x7f1d5c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:23.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.647+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c03f030 con 0x7f1d68103ff0 2024-09-12T03:26:23.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.647+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c004440 con 0x7f1d68103ff0 2024-09-12T03:26:23.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.647+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c035070 con 0x7f1d68103ff0 2024-09-12T03:26:23.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.647+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6819f840 con 0x7f1d68103ff0 2024-09-12T03:26:23.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.647+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6819fcd0 con 0x7f1d68103ff0 2024-09-12T03:26:23.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.649+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d681043d0 con 0x7f1d68103ff0 2024-09-12T03:26:23.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.651+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d5c042030 con 0x7f1d68103ff0 2024-09-12T03:26:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.653+0000 7f1d62a00640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d500a2c90 0x7f1d500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.653+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1d5c0ef950 con 0x7f1d68103ff0 2024-09-12T03:26:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.653+0000 7f1d6d400640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d500a2c90 0x7f1d500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:23.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.654+0000 7f1d6d400640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d500a2c90 0x7f1d500a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1d680fff90 tx=0x7f1d58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:23.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.658+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d5c0bc400 con 0x7f1d68103ff0 2024-09-12T03:26:23.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.841+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d68062630 con 0x7f1d500a2c90 2024-09-12T03:26:23.843 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:23 smithi050 ceph-mon[30378]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:26:23.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.850+0000 7f1d62a00640 1 -- 172.21.15.50:0/1135513344 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340373 (secure 0 0 0) 0x7f1d68062630 con 0x7f1d500a2c90 2024-09-12T03:26:23.855 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:23.856 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d500a2c90 msgr2=0x7f1d500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d500a2c90 0x7f1d500a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1d680fff90 tx=0x7f1d58005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 msgr2=0x7f1d6819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d6819a600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c006010 tx=0x7f1d5c004040 comp rx=0 tx=0).stop 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 shutdown_connections 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f1d500a2c90 0x7f1d500a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d68109120 0x7f1d6819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d681049a0 0x7f1d6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 --2- 172.21.15.50:0/1135513344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d68103ff0 0x7f1d6819a600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 >> 172.21.15.50:0/1135513344 conn(0x7f1d680ffa90 msgr2=0x7f1d68100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 shutdown_connections 2024-09-12T03:26:23.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:23.855+0000 7f1d70800640 1 -- 172.21.15.50:0/1135513344 wait complete. 2024-09-12T03:26:24.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:23 smithi142 ceph-mon[35329]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:26:24.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:23 smithi082 ceph-mon[35046]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:26:24.664 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-12T03:26:22.773413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":504,"num_read_kb":373,"num_write":352,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180870,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376532,"kb_used_data":14720,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565292,"statfs":{"total":1151940427776,"available":1151554859008,"internally_reserved":0,"allocated":15073280,"data_stored":8416661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":24,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.115820"},"pg_stats":[{"pgid":"2.1f","version":"92'9","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660847+0000","last_change":"2024-09-12T03:24:29.302864+0000","last_active":"2024-09-12T03:26:14.660847+0000","last_peered":"2024-09-12T03:26:14.660847+0000","last_clean":"2024-09-12T03:26:14.660847+0000","last_became_active":"2024-09-12T03:24:29.302551+0000","last_became_peered":"2024-09-12T03:24:29.302551+0000","last_unstale":"2024-09-12T03:26:14.660847+0000","last_undegraded":"2024-09-12T03:26:14.660847+0000","last_fullsized":"2024-09-12T03:26:14.660847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:10.151772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685643+0000","last_change":"2024-09-12T03:24:31.649557+0000","last_active":"2024-09-12T03:26:14.685643+0000","last_peered":"2024-09-12T03:26:14.685643+0000","last_clean":"2024-09-12T03:26:14.685643+0000","last_became_active":"2024-09-12T03:24:31.649082+0000","last_became_peered":"2024-09-12T03:24:31.649082+0000","last_unstale":"2024-09-12T03:26:14.685643+0000","last_undegraded":"2024-09-12T03:26:14.685643+0000","last_fullsized":"2024-09-12T03:26:14.685643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:54.245063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663273+0000","last_change":"2024-09-12T03:24:40.708670+0000","last_active":"2024-09-12T03:26:14.663273+0000","last_peered":"2024-09-12T03:26:14.663273+0000","last_clean":"2024-09-12T03:26:14.663273+0000","last_became_active":"2024-09-12T03:24:40.708264+0000","last_became_peered":"2024-09-12T03:24:40.708264+0000","last_unstale":"2024-09-12T03:26:14.663273+0000","last_undegraded":"2024-09-12T03:26:14.663273+0000","last_fullsized":"2024-09-12T03:26:14.663273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:00.670492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685839+0000","last_change":"2024-09-12T03:24:29.312254+0000","last_active":"2024-09-12T03:26:14.685839+0000","last_peered":"2024-09-12T03:26:14.685839+0000","last_clean":"2024-09-12T03:26:14.685839+0000","last_became_active":"2024-09-12T03:24:29.311842+0000","last_became_peered":"2024-09-12T03:24:29.311842+0000","last_unstale":"2024-09-12T03:26:14.685839+0000","last_undegraded":"2024-09-12T03:26:14.685839+0000","last_fullsized":"2024-09-12T03:26:14.685839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:43.298456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669016+0000","last_change":"2024-09-12T03:24:31.644330+0000","last_active":"2024-09-12T03:26:14.669016+0000","last_peered":"2024-09-12T03:26:14.669016+0000","last_clean":"2024-09-12T03:26:14.669016+0000","last_became_active":"2024-09-12T03:24:31.642904+0000","last_became_peered":"2024-09-12T03:24:31.642904+0000","last_unstale":"2024-09-12T03:26:14.669016+0000","last_undegraded":"2024-09-12T03:26:14.669016+0000","last_fullsized":"2024-09-12T03:26:14.669016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:14.304333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669082+0000","last_change":"2024-09-12T03:24:40.706682+0000","last_active":"2024-09-12T03:26:14.669082+0000","last_peered":"2024-09-12T03:26:14.669082+0000","last_clean":"2024-09-12T03:26:14.669082+0000","last_became_active":"2024-09-12T03:24:40.706143+0000","last_became_peered":"2024-09-12T03:24:40.706143+0000","last_unstale":"2024-09-12T03:26:14.669082+0000","last_undegraded":"2024-09-12T03:26:14.669082+0000","last_fullsized":"2024-09-12T03:26:14.669082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:05.146486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665732+0000","last_change":"2024-09-12T03:24:29.307621+0000","last_active":"2024-09-12T03:26:14.665732+0000","last_peered":"2024-09-12T03:26:14.665732+0000","last_clean":"2024-09-12T03:26:14.665732+0000","last_became_active":"2024-09-12T03:24:29.307258+0000","last_became_peered":"2024-09-12T03:24:29.307258+0000","last_unstale":"2024-09-12T03:26:14.665732+0000","last_undegraded":"2024-09-12T03:26:14.665732+0000","last_fullsized":"2024-09-12T03:26:14.665732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:03.513367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670424+0000","last_change":"2024-09-12T03:24:31.649972+0000","last_active":"2024-09-12T03:26:14.670424+0000","last_peered":"2024-09-12T03:26:14.670424+0000","last_clean":"2024-09-12T03:26:14.670424+0000","last_became_active":"2024-09-12T03:24:31.649693+0000","last_became_peered":"2024-09-12T03:24:31.649693+0000","last_unstale":"2024-09-12T03:26:14.670424+0000","last_undegraded":"2024-09-12T03:26:14.670424+0000","last_fullsized":"2024-09-12T03:26:14.670424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:10.502054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669572+0000","last_change":"2024-09-12T03:24:40.698995+0000","last_active":"2024-09-12T03:26:14.669572+0000","last_peered":"2024-09-12T03:26:14.669572+0000","last_clean":"2024-09-12T03:26:14.669572+0000","last_became_active":"2024-09-12T03:24:40.698094+0000","last_became_peered":"2024-09-12T03:24:40.698094+0000","last_unstale":"2024-09-12T03:26:14.669572+0000","last_undegraded":"2024-09-12T03:26:14.669572+0000","last_fullsized":"2024-09-12T03:26:14.669572+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:39.266209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.675457+0000","last_change":"2024-09-12T03:26:14.674955+0000","last_active":"2024-09-12T03:26:14.675457+0000","last_peered":"2024-09-12T03:26:14.675457+0000","last_clean":"2024-09-12T03:26:14.675457+0000","last_became_active":"2024-09-12T03:26:14.671862+0000","last_became_peered":"2024-09-12T03:26:14.671862+0000","last_unstale":"2024-09-12T03:26:14.675457+0000","last_undegraded":"2024-09-12T03:26:14.675457+0000","last_fullsized":"2024-09-12T03:26:14.675457+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:57.427928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670977+0000","last_change":"2024-09-12T03:24:31.648735+0000","last_active":"2024-09-12T03:26:14.670977+0000","last_peered":"2024-09-12T03:26:14.670977+0000","last_clean":"2024-09-12T03:26:14.670977+0000","last_became_active":"2024-09-12T03:24:31.648416+0000","last_became_peered":"2024-09-12T03:24:31.648416+0000","last_unstale":"2024-09-12T03:26:14.670977+0000","last_undegraded":"2024-09-12T03:26:14.670977+0000","last_fullsized":"2024-09-12T03:26:14.670977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:24.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669151+0000","last_change":"2024-09-12T03:24:40.699941+0000","last_active":"2024-09-12T03:26:14.669151+0000","last_peered":"2024-09-12T03:26:14.669151+0000","last_clean":"2024-09-12T03:26:14.669151+0000","last_became_active":"2024-09-12T03:24:40.699126+0000","last_became_peered":"2024-09-12T03:24:40.699126+0000","last_unstale":"2024-09-12T03:26:14.669151+0000","last_undegraded":"2024-09-12T03:26:14.669151+0000","last_fullsized":"2024-09-12T03:26:14.669151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:24.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684307+0000","last_change":"2024-09-12T03:24:31.650650+0000","last_active":"2024-09-12T03:26:14.684307+0000","last_peered":"2024-09-12T03:26:14.684307+0000","last_clean":"2024-09-12T03:26:14.684307+0000","last_became_active":"2024-09-12T03:24:31.650279+0000","last_became_peered":"2024-09-12T03:24:31.650279+0000","last_unstale":"2024-09-12T03:26:14.684307+0000","last_undegraded":"2024-09-12T03:26:14.684307+0000","last_fullsized":"2024-09-12T03:26:14.684307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:49.860317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665270+0000","last_change":"2024-09-12T03:24:29.307426+0000","last_active":"2024-09-12T03:26:14.665270+0000","last_peered":"2024-09-12T03:26:14.665270+0000","last_clean":"2024-09-12T03:26:14.665270+0000","last_became_active":"2024-09-12T03:24:29.307119+0000","last_became_peered":"2024-09-12T03:24:29.307119+0000","last_unstale":"2024-09-12T03:26:14.665270+0000","last_undegraded":"2024-09-12T03:26:14.665270+0000","last_fullsized":"2024-09-12T03:26:14.665270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:48.957594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670467+0000","last_change":"2024-09-12T03:24:40.700292+0000","last_active":"2024-09-12T03:26:14.670467+0000","last_peered":"2024-09-12T03:26:14.670467+0000","last_clean":"2024-09-12T03:26:14.670467+0000","last_became_active":"2024-09-12T03:24:40.699819+0000","last_became_peered":"2024-09-12T03:24:40.699819+0000","last_unstale":"2024-09-12T03:26:14.670467+0000","last_undegraded":"2024-09-12T03:26:14.670467+0000","last_fullsized":"2024-09-12T03:26:14.670467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:43.022322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685437+0000","last_change":"2024-09-12T03:24:31.647669+0000","last_active":"2024-09-12T03:26:14.685437+0000","last_peered":"2024-09-12T03:26:14.685437+0000","last_clean":"2024-09-12T03:26:14.685437+0000","last_became_active":"2024-09-12T03:24:31.644429+0000","last_became_peered":"2024-09-12T03:24:31.644429+0000","last_unstale":"2024-09-12T03:26:14.685437+0000","last_undegraded":"2024-09-12T03:26:14.685437+0000","last_fullsized":"2024-09-12T03:26:14.685437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:22.556399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663435+0000","last_change":"2024-09-12T03:24:29.309334+0000","last_active":"2024-09-12T03:26:14.663435+0000","last_peered":"2024-09-12T03:26:14.663435+0000","last_clean":"2024-09-12T03:26:14.663435+0000","last_became_active":"2024-09-12T03:24:29.309007+0000","last_became_peered":"2024-09-12T03:24:29.309007+0000","last_unstale":"2024-09-12T03:26:14.663435+0000","last_undegraded":"2024-09-12T03:26:14.663435+0000","last_fullsized":"2024-09-12T03:26:14.663435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:40.898104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669668+0000","last_change":"2024-09-12T03:24:40.706611+0000","last_active":"2024-09-12T03:26:14.669668+0000","last_peered":"2024-09-12T03:26:14.669668+0000","last_clean":"2024-09-12T03:26:14.669668+0000","last_became_active":"2024-09-12T03:24:40.706058+0000","last_became_peered":"2024-09-12T03:24:40.706058+0000","last_unstale":"2024-09-12T03:26:14.669668+0000","last_undegraded":"2024-09-12T03:26:14.669668+0000","last_fullsized":"2024-09-12T03:26:14.669668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:16.547894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667191+0000","last_change":"2024-09-12T03:24:31.648458+0000","last_active":"2024-09-12T03:26:14.667191+0000","last_peered":"2024-09-12T03:26:14.667191+0000","last_clean":"2024-09-12T03:26:14.667191+0000","last_became_active":"2024-09-12T03:24:31.644171+0000","last_became_peered":"2024-09-12T03:24:31.644171+0000","last_unstale":"2024-09-12T03:26:14.667191+0000","last_undegraded":"2024-09-12T03:26:14.667191+0000","last_fullsized":"2024-09-12T03:26:14.667191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:20.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685145+0000","last_change":"2024-09-12T03:24:29.309013+0000","last_active":"2024-09-12T03:26:14.685145+0000","last_peered":"2024-09-12T03:26:14.685145+0000","last_clean":"2024-09-12T03:26:14.685145+0000","last_became_active":"2024-09-12T03:24:29.308459+0000","last_became_peered":"2024-09-12T03:24:29.308459+0000","last_unstale":"2024-09-12T03:26:14.685145+0000","last_undegraded":"2024-09-12T03:26:14.685145+0000","last_fullsized":"2024-09-12T03:26:14.685145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:38.655615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665514+0000","last_change":"2024-09-12T03:25:14.321079+0000","last_active":"2024-09-12T03:26:14.665514+0000","last_peered":"2024-09-12T03:26:14.665514+0000","last_clean":"2024-09-12T03:26:14.665514+0000","last_became_active":"2024-09-12T03:25:14.320602+0000","last_became_peered":"2024-09-12T03:25:14.320602+0000","last_unstale":"2024-09-12T03:26:14.665514+0000","last_undegraded":"2024-09-12T03:26:14.665514+0000","last_fullsized":"2024-09-12T03:26:14.665514+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:18.787689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665017+0000","last_change":"2024-09-12T03:24:31.637560+0000","last_active":"2024-09-12T03:26:14.665017+0000","last_peered":"2024-09-12T03:26:14.665017+0000","last_clean":"2024-09-12T03:26:14.665017+0000","last_became_active":"2024-09-12T03:24:31.636962+0000","last_became_peered":"2024-09-12T03:24:31.636962+0000","last_unstale":"2024-09-12T03:26:14.665017+0000","last_undegraded":"2024-09-12T03:26:14.665017+0000","last_fullsized":"2024-09-12T03:26:14.665017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:44.468397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670183+0000","last_change":"2024-09-12T03:24:29.305262+0000","last_active":"2024-09-12T03:26:14.670183+0000","last_peered":"2024-09-12T03:26:14.670183+0000","last_clean":"2024-09-12T03:26:14.670183+0000","last_became_active":"2024-09-12T03:24:29.304937+0000","last_became_peered":"2024-09-12T03:24:29.304937+0000","last_unstale":"2024-09-12T03:26:14.670183+0000","last_undegraded":"2024-09-12T03:26:14.670183+0000","last_fullsized":"2024-09-12T03:26:14.670183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:45.172526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670378+0000","last_change":"2024-09-12T03:24:40.702517+0000","last_active":"2024-09-12T03:26:14.670378+0000","last_peered":"2024-09-12T03:26:14.670378+0000","last_clean":"2024-09-12T03:26:14.670378+0000","last_became_active":"2024-09-12T03:24:40.702093+0000","last_became_peered":"2024-09-12T03:24:40.702093+0000","last_unstale":"2024-09-12T03:26:14.670378+0000","last_undegraded":"2024-09-12T03:26:14.670378+0000","last_fullsized":"2024-09-12T03:26:14.670378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:00.817214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668754+0000","last_change":"2024-09-12T03:24:31.650541+0000","last_active":"2024-09-12T03:26:14.668754+0000","last_peered":"2024-09-12T03:26:14.668754+0000","last_clean":"2024-09-12T03:26:14.668754+0000","last_became_active":"2024-09-12T03:24:31.650180+0000","last_became_peered":"2024-09-12T03:24:31.650180+0000","last_unstale":"2024-09-12T03:26:14.668754+0000","last_undegraded":"2024-09-12T03:26:14.668754+0000","last_fullsized":"2024-09-12T03:26:14.668754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:11.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670065+0000","last_change":"2024-09-12T03:24:29.335888+0000","last_active":"2024-09-12T03:26:14.670065+0000","last_peered":"2024-09-12T03:26:14.670065+0000","last_clean":"2024-09-12T03:26:14.670065+0000","last_became_active":"2024-09-12T03:24:29.335535+0000","last_became_peered":"2024-09-12T03:24:29.335535+0000","last_unstale":"2024-09-12T03:26:14.670065+0000","last_undegraded":"2024-09-12T03:26:14.670065+0000","last_fullsized":"2024-09-12T03:26:14.670065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:19.246874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663288+0000","last_change":"2024-09-12T03:24:40.696385+0000","last_active":"2024-09-12T03:26:14.663288+0000","last_peered":"2024-09-12T03:26:14.663288+0000","last_clean":"2024-09-12T03:26:14.663288+0000","last_became_active":"2024-09-12T03:24:40.695968+0000","last_became_peered":"2024-09-12T03:24:40.695968+0000","last_unstale":"2024-09-12T03:26:14.663288+0000","last_undegraded":"2024-09-12T03:26:14.663288+0000","last_fullsized":"2024-09-12T03:26:14.663288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:31.768769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.686005+0000","last_change":"2024-09-12T03:24:31.655017+0000","last_active":"2024-09-12T03:26:14.686005+0000","last_peered":"2024-09-12T03:26:14.686005+0000","last_clean":"2024-09-12T03:26:14.686005+0000","last_became_active":"2024-09-12T03:24:31.654597+0000","last_became_peered":"2024-09-12T03:24:31.654597+0000","last_unstale":"2024-09-12T03:26:14.686005+0000","last_undegraded":"2024-09-12T03:26:14.686005+0000","last_fullsized":"2024-09-12T03:26:14.686005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:13.053238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660720+0000","last_change":"2024-09-12T03:24:29.303991+0000","last_active":"2024-09-12T03:26:14.660720+0000","last_peered":"2024-09-12T03:26:14.660720+0000","last_clean":"2024-09-12T03:26:14.660720+0000","last_became_active":"2024-09-12T03:24:29.303682+0000","last_became_peered":"2024-09-12T03:24:29.303682+0000","last_unstale":"2024-09-12T03:26:14.660720+0000","last_undegraded":"2024-09-12T03:26:14.660720+0000","last_fullsized":"2024-09-12T03:26:14.660720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:17:59.403461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684749+0000","last_change":"2024-09-12T03:24:40.704304+0000","last_active":"2024-09-12T03:26:14.684749+0000","last_peered":"2024-09-12T03:26:14.684749+0000","last_clean":"2024-09-12T03:26:14.684749+0000","last_became_active":"2024-09-12T03:24:40.703967+0000","last_became_peered":"2024-09-12T03:24:40.703967+0000","last_unstale":"2024-09-12T03:26:14.684749+0000","last_undegraded":"2024-09-12T03:26:14.684749+0000","last_fullsized":"2024-09-12T03:26:14.684749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:00.337853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684960+0000","last_change":"2024-09-12T03:24:31.650303+0000","last_active":"2024-09-12T03:26:14.684960+0000","last_peered":"2024-09-12T03:26:14.684960+0000","last_clean":"2024-09-12T03:26:14.684960+0000","last_became_active":"2024-09-12T03:24:31.649223+0000","last_became_peered":"2024-09-12T03:24:31.649223+0000","last_unstale":"2024-09-12T03:26:14.684960+0000","last_undegraded":"2024-09-12T03:26:14.684960+0000","last_fullsized":"2024-09-12T03:26:14.684960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:45.190363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665953+0000","last_change":"2024-09-12T03:24:29.310570+0000","last_active":"2024-09-12T03:26:14.665953+0000","last_peered":"2024-09-12T03:26:14.665953+0000","last_clean":"2024-09-12T03:26:14.665953+0000","last_became_active":"2024-09-12T03:24:29.310249+0000","last_became_peered":"2024-09-12T03:24:29.310249+0000","last_unstale":"2024-09-12T03:26:14.665953+0000","last_undegraded":"2024-09-12T03:26:14.665953+0000","last_fullsized":"2024-09-12T03:26:14.665953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.049566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667328+0000","last_change":"2024-09-12T03:24:40.696716+0000","last_active":"2024-09-12T03:26:14.667328+0000","last_peered":"2024-09-12T03:26:14.667328+0000","last_clean":"2024-09-12T03:26:14.667328+0000","last_became_active":"2024-09-12T03:24:40.696322+0000","last_became_peered":"2024-09-12T03:24:40.696322+0000","last_unstale":"2024-09-12T03:26:14.667328+0000","last_undegraded":"2024-09-12T03:26:14.667328+0000","last_fullsized":"2024-09-12T03:26:14.667328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:31.315393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663998+0000","last_change":"2024-09-12T03:24:31.652360+0000","last_active":"2024-09-12T03:26:14.663998+0000","last_peered":"2024-09-12T03:26:14.663998+0000","last_clean":"2024-09-12T03:26:14.663998+0000","last_became_active":"2024-09-12T03:24:31.652066+0000","last_became_peered":"2024-09-12T03:24:31.652066+0000","last_unstale":"2024-09-12T03:26:14.663998+0000","last_undegraded":"2024-09-12T03:26:14.663998+0000","last_fullsized":"2024-09-12T03:26:14.663998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:54:15.312876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660586+0000","last_change":"2024-09-12T03:24:29.305811+0000","last_active":"2024-09-12T03:26:14.660586+0000","last_peered":"2024-09-12T03:26:14.660586+0000","last_clean":"2024-09-12T03:26:14.660586+0000","last_became_active":"2024-09-12T03:24:29.305531+0000","last_became_peered":"2024-09-12T03:24:29.305531+0000","last_unstale":"2024-09-12T03:26:14.660586+0000","last_undegraded":"2024-09-12T03:26:14.660586+0000","last_fullsized":"2024-09-12T03:26:14.660586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:08.496670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665042+0000","last_change":"2024-09-12T03:25:14.267079+0000","last_active":"2024-09-12T03:26:14.665042+0000","last_peered":"2024-09-12T03:26:14.665042+0000","last_clean":"2024-09-12T03:26:14.665042+0000","last_became_active":"2024-09-12T03:25:14.266625+0000","last_became_peered":"2024-09-12T03:25:14.266625+0000","last_unstale":"2024-09-12T03:26:14.665042+0000","last_undegraded":"2024-09-12T03:26:14.665042+0000","last_fullsized":"2024-09-12T03:26:14.665042+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663729+0000","last_change":"2024-09-12T03:25:13.812997+0000","last_active":"2024-09-12T03:26:14.663729+0000","last_peered":"2024-09-12T03:26:14.663729+0000","last_clean":"2024-09-12T03:26:14.663729+0000","last_became_active":"2024-09-12T03:25:13.812596+0000","last_became_peered":"2024-09-12T03:25:13.812596+0000","last_unstale":"2024-09-12T03:26:14.663729+0000","last_undegraded":"2024-09-12T03:26:14.663729+0000","last_fullsized":"2024-09-12T03:26:14.663729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:43.228416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684433+0000","last_change":"2024-09-12T03:24:29.301269+0000","last_active":"2024-09-12T03:26:14.684433+0000","last_peered":"2024-09-12T03:26:14.684433+0000","last_clean":"2024-09-12T03:26:14.684433+0000","last_became_active":"2024-09-12T03:24:29.300937+0000","last_became_peered":"2024-09-12T03:24:29.300937+0000","last_unstale":"2024-09-12T03:26:14.684433+0000","last_undegraded":"2024-09-12T03:26:14.684433+0000","last_fullsized":"2024-09-12T03:26:14.684433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:24.453709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684828+0000","last_change":"2024-09-12T03:24:40.708631+0000","last_active":"2024-09-12T03:26:14.684828+0000","last_peered":"2024-09-12T03:26:14.684828+0000","last_clean":"2024-09-12T03:26:14.684828+0000","last_became_active":"2024-09-12T03:24:40.706381+0000","last_became_peered":"2024-09-12T03:24:40.706381+0000","last_unstale":"2024-09-12T03:26:14.684828+0000","last_undegraded":"2024-09-12T03:26:14.684828+0000","last_fullsized":"2024-09-12T03:26:14.684828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:10.945538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670344+0000","last_change":"2024-09-12T03:24:31.652471+0000","last_active":"2024-09-12T03:26:14.670344+0000","last_peered":"2024-09-12T03:26:14.670344+0000","last_clean":"2024-09-12T03:26:14.670344+0000","last_became_active":"2024-09-12T03:24:31.651566+0000","last_became_peered":"2024-09-12T03:24:31.651566+0000","last_unstale":"2024-09-12T03:26:14.670344+0000","last_undegraded":"2024-09-12T03:26:14.670344+0000","last_fullsized":"2024-09-12T03:26:14.670344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:01.238554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":98,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:15.943456+0000","last_change":"2024-09-12T03:24:29.306214+0000","last_active":"2024-09-12T03:26:15.943456+0000","last_peered":"2024-09-12T03:26:15.943456+0000","last_clean":"2024-09-12T03:26:15.943456+0000","last_became_active":"2024-09-12T03:24:29.305846+0000","last_became_peered":"2024-09-12T03:24:29.305846+0000","last_unstale":"2024-09-12T03:26:15.943456+0000","last_undegraded":"2024-09-12T03:26:15.943456+0000","last_fullsized":"2024-09-12T03:26:15.943456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:50.940377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685256+0000","last_change":"2024-09-12T03:24:40.709412+0000","last_active":"2024-09-12T03:26:14.685256+0000","last_peered":"2024-09-12T03:26:14.685256+0000","last_clean":"2024-09-12T03:26:14.685256+0000","last_became_active":"2024-09-12T03:24:40.705106+0000","last_became_peered":"2024-09-12T03:24:40.705106+0000","last_unstale":"2024-09-12T03:26:14.685256+0000","last_undegraded":"2024-09-12T03:26:14.685256+0000","last_fullsized":"2024-09-12T03:26:14.685256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:32.963060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684523+0000","last_change":"2024-09-12T03:24:31.644654+0000","last_active":"2024-09-12T03:26:14.684523+0000","last_peered":"2024-09-12T03:26:14.684523+0000","last_clean":"2024-09-12T03:26:14.684523+0000","last_became_active":"2024-09-12T03:24:31.642477+0000","last_became_peered":"2024-09-12T03:24:31.642477+0000","last_unstale":"2024-09-12T03:26:14.684523+0000","last_undegraded":"2024-09-12T03:26:14.684523+0000","last_fullsized":"2024-09-12T03:26:14.684523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.758784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684214+0000","last_change":"2024-09-12T03:24:29.306850+0000","last_active":"2024-09-12T03:26:14.684214+0000","last_peered":"2024-09-12T03:26:14.684214+0000","last_clean":"2024-09-12T03:26:14.684214+0000","last_became_active":"2024-09-12T03:24:29.305902+0000","last_became_peered":"2024-09-12T03:24:29.305902+0000","last_unstale":"2024-09-12T03:26:14.684214+0000","last_undegraded":"2024-09-12T03:26:14.684214+0000","last_fullsized":"2024-09-12T03:26:14.684214+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:02.564521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670180+0000","last_change":"2024-09-12T03:24:40.700966+0000","last_active":"2024-09-12T03:26:14.670180+0000","last_peered":"2024-09-12T03:26:14.670180+0000","last_clean":"2024-09-12T03:26:14.670180+0000","last_became_active":"2024-09-12T03:24:40.700553+0000","last_became_peered":"2024-09-12T03:24:40.700553+0000","last_unstale":"2024-09-12T03:26:14.670180+0000","last_undegraded":"2024-09-12T03:26:14.670180+0000","last_fullsized":"2024-09-12T03:26:14.670180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:14.855459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685372+0000","last_change":"2024-09-12T03:24:31.653431+0000","last_active":"2024-09-12T03:26:14.685372+0000","last_peered":"2024-09-12T03:26:14.685372+0000","last_clean":"2024-09-12T03:26:14.685372+0000","last_became_active":"2024-09-12T03:24:31.648771+0000","last_became_peered":"2024-09-12T03:24:31.648771+0000","last_unstale":"2024-09-12T03:26:14.685372+0000","last_undegraded":"2024-09-12T03:26:14.685372+0000","last_fullsized":"2024-09-12T03:26:14.685372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:54.052828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669709+0000","last_change":"2024-09-12T03:24:29.305825+0000","last_active":"2024-09-12T03:26:14.669709+0000","last_peered":"2024-09-12T03:26:14.669709+0000","last_clean":"2024-09-12T03:26:14.669709+0000","last_became_active":"2024-09-12T03:24:29.305468+0000","last_became_peered":"2024-09-12T03:24:29.305468+0000","last_unstale":"2024-09-12T03:26:14.669709+0000","last_undegraded":"2024-09-12T03:26:14.669709+0000","last_fullsized":"2024-09-12T03:26:14.669709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:30.899307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":368,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684290+0000","last_change":"2024-09-12T03:23:13.507232+0000","last_active":"2024-09-12T03:26:14.684290+0000","last_peered":"2024-09-12T03:26:14.684290+0000","last_clean":"2024-09-12T03:26:14.684290+0000","last_became_active":"2024-09-12T03:23:13.506683+0000","last_became_peered":"2024-09-12T03:23:13.506683+0000","last_unstale":"2024-09-12T03:26:14.684290+0000","last_undegraded":"2024-09-12T03:26:14.684290+0000","last_fullsized":"2024-09-12T03:26:14.684290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:33.594997+0000","last_clean_scrub_stamp":"2024-09-12T03:19:33.594997+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.684658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684664+0000","last_change":"2024-09-12T03:24:40.696812+0000","last_active":"2024-09-12T03:26:14.684664+0000","last_peered":"2024-09-12T03:26:14.684664+0000","last_clean":"2024-09-12T03:26:14.684664+0000","last_became_active":"2024-09-12T03:24:40.695586+0000","last_became_peered":"2024-09-12T03:24:40.695586+0000","last_unstale":"2024-09-12T03:26:14.684664+0000","last_undegraded":"2024-09-12T03:26:14.684664+0000","last_fullsized":"2024-09-12T03:26:14.684664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:10.086113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684668+0000","last_change":"2024-09-12T03:24:31.644517+0000","last_active":"2024-09-12T03:26:14.684668+0000","last_peered":"2024-09-12T03:26:14.684668+0000","last_clean":"2024-09-12T03:26:14.684668+0000","last_became_active":"2024-09-12T03:24:31.643996+0000","last_became_peered":"2024-09-12T03:24:31.643996+0000","last_unstale":"2024-09-12T03:26:14.684668+0000","last_undegraded":"2024-09-12T03:26:14.684668+0000","last_fullsized":"2024-09-12T03:26:14.684668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:57.543397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669418+0000","last_change":"2024-09-12T03:25:14.266550+0000","last_active":"2024-09-12T03:26:14.669418+0000","last_peered":"2024-09-12T03:26:14.669418+0000","last_clean":"2024-09-12T03:26:14.669418+0000","last_became_active":"2024-09-12T03:25:14.266202+0000","last_became_peered":"2024-09-12T03:25:14.266202+0000","last_unstale":"2024-09-12T03:26:14.669418+0000","last_undegraded":"2024-09-12T03:26:14.669418+0000","last_fullsized":"2024-09-12T03:26:14.669418+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:00.981147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670158+0000","last_change":"2024-09-12T03:24:40.697001+0000","last_active":"2024-09-12T03:26:14.670158+0000","last_peered":"2024-09-12T03:26:14.670158+0000","last_clean":"2024-09-12T03:26:14.670158+0000","last_became_active":"2024-09-12T03:24:40.696091+0000","last_became_peered":"2024-09-12T03:24:40.696091+0000","last_unstale":"2024-09-12T03:26:14.670158+0000","last_undegraded":"2024-09-12T03:26:14.670158+0000","last_fullsized":"2024-09-12T03:26:14.670158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:07.073034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684396+0000","last_change":"2024-09-12T03:24:31.655819+0000","last_active":"2024-09-12T03:26:14.684396+0000","last_peered":"2024-09-12T03:26:14.684396+0000","last_clean":"2024-09-12T03:26:14.684396+0000","last_became_active":"2024-09-12T03:24:31.655417+0000","last_became_peered":"2024-09-12T03:24:31.655417+0000","last_unstale":"2024-09-12T03:26:14.684396+0000","last_undegraded":"2024-09-12T03:26:14.684396+0000","last_fullsized":"2024-09-12T03:26:14.684396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:54.380047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670788+0000","last_change":"2024-09-12T03:24:29.303732+0000","last_active":"2024-09-12T03:26:14.670788+0000","last_peered":"2024-09-12T03:26:14.670788+0000","last_clean":"2024-09-12T03:26:14.670788+0000","last_became_active":"2024-09-12T03:24:29.302314+0000","last_became_peered":"2024-09-12T03:24:29.302314+0000","last_unstale":"2024-09-12T03:26:14.670788+0000","last_undegraded":"2024-09-12T03:26:14.670788+0000","last_fullsized":"2024-09-12T03:26:14.670788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:51.555510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668153+0000","last_change":"2024-09-12T03:25:13.817846+0000","last_active":"2024-09-12T03:26:14.668153+0000","last_peered":"2024-09-12T03:26:14.668153+0000","last_clean":"2024-09-12T03:26:14.668153+0000","last_became_active":"2024-09-12T03:25:13.817102+0000","last_became_peered":"2024-09-12T03:25:13.817102+0000","last_unstale":"2024-09-12T03:26:14.668153+0000","last_undegraded":"2024-09-12T03:26:14.668153+0000","last_fullsized":"2024-09-12T03:26:14.668153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:26.472731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684558+0000","last_change":"2024-09-12T03:24:31.645293+0000","last_active":"2024-09-12T03:26:14.684558+0000","last_peered":"2024-09-12T03:26:14.684558+0000","last_clean":"2024-09-12T03:26:14.684558+0000","last_became_active":"2024-09-12T03:24:31.644944+0000","last_became_peered":"2024-09-12T03:24:31.644944+0000","last_unstale":"2024-09-12T03:26:14.684558+0000","last_undegraded":"2024-09-12T03:26:14.684558+0000","last_fullsized":"2024-09-12T03:26:14.684558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:38.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685463+0000","last_change":"2024-09-12T03:24:29.302710+0000","last_active":"2024-09-12T03:26:14.685463+0000","last_peered":"2024-09-12T03:26:14.685463+0000","last_clean":"2024-09-12T03:26:14.685463+0000","last_became_active":"2024-09-12T03:24:29.302262+0000","last_became_peered":"2024-09-12T03:24:29.302262+0000","last_unstale":"2024-09-12T03:26:14.685463+0000","last_undegraded":"2024-09-12T03:26:14.685463+0000","last_fullsized":"2024-09-12T03:26:14.685463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:08.392390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669977+0000","last_change":"2024-09-12T03:24:40.696164+0000","last_active":"2024-09-12T03:26:14.669977+0000","last_peered":"2024-09-12T03:26:14.669977+0000","last_clean":"2024-09-12T03:26:14.669977+0000","last_became_active":"2024-09-12T03:24:40.695786+0000","last_became_peered":"2024-09-12T03:24:40.695786+0000","last_unstale":"2024-09-12T03:26:14.669977+0000","last_undegraded":"2024-09-12T03:26:14.669977+0000","last_fullsized":"2024-09-12T03:26:14.669977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:27.063196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667558+0000","last_change":"2024-09-12T03:24:31.645341+0000","last_active":"2024-09-12T03:26:14.667558+0000","last_peered":"2024-09-12T03:26:14.667558+0000","last_clean":"2024-09-12T03:26:14.667558+0000","last_became_active":"2024-09-12T03:24:31.644496+0000","last_became_peered":"2024-09-12T03:24:31.644496+0000","last_unstale":"2024-09-12T03:26:14.667558+0000","last_undegraded":"2024-09-12T03:26:14.667558+0000","last_fullsized":"2024-09-12T03:26:14.667558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:09.416806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.679152+0000","last_change":"2024-09-12T03:26:14.678640+0000","last_active":"2024-09-12T03:26:14.679152+0000","last_peered":"2024-09-12T03:26:14.679152+0000","last_clean":"2024-09-12T03:26:14.679152+0000","last_became_active":"2024-09-12T03:26:14.671800+0000","last_became_peered":"2024-09-12T03:26:14.671800+0000","last_unstale":"2024-09-12T03:26:14.679152+0000","last_undegraded":"2024-09-12T03:26:14.679152+0000","last_fullsized":"2024-09-12T03:26:14.679152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:40.858472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670432+0000","last_change":"2024-09-12T03:24:40.703795+0000","last_active":"2024-09-12T03:26:14.670432+0000","last_peered":"2024-09-12T03:26:14.670432+0000","last_clean":"2024-09-12T03:26:14.670432+0000","last_became_active":"2024-09-12T03:24:40.703361+0000","last_became_peered":"2024-09-12T03:24:40.703361+0000","last_unstale":"2024-09-12T03:26:14.670432+0000","last_undegraded":"2024-09-12T03:26:14.670432+0000","last_fullsized":"2024-09-12T03:26:14.670432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:12.430431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670311+0000","last_change":"2024-09-12T03:24:31.649521+0000","last_active":"2024-09-12T03:26:14.670311+0000","last_peered":"2024-09-12T03:26:14.670311+0000","last_clean":"2024-09-12T03:26:14.670311+0000","last_became_active":"2024-09-12T03:24:31.649217+0000","last_became_peered":"2024-09-12T03:24:31.649217+0000","last_unstale":"2024-09-12T03:26:14.670311+0000","last_undegraded":"2024-09-12T03:26:14.670311+0000","last_fullsized":"2024-09-12T03:26:14.670311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:33.885153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":79,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685820+0000","last_change":"2024-09-12T03:26:14.685820+0000","last_active":"2024-09-12T03:26:14.685820+0000","last_peered":"2024-09-12T03:26:14.685820+0000","last_clean":"2024-09-12T03:26:14.685820+0000","last_became_active":"2024-09-12T03:26:14.672467+0000","last_became_peered":"2024-09-12T03:26:14.672467+0000","last_unstale":"2024-09-12T03:26:14.685820+0000","last_undegraded":"2024-09-12T03:26:14.685820+0000","last_fullsized":"2024-09-12T03:26:14.685820+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:58.530737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685256+0000","last_change":"2024-09-12T03:24:40.707719+0000","last_active":"2024-09-12T03:26:14.685256+0000","last_peered":"2024-09-12T03:26:14.685256+0000","last_clean":"2024-09-12T03:26:14.685256+0000","last_became_active":"2024-09-12T03:24:40.707364+0000","last_became_peered":"2024-09-12T03:24:40.707364+0000","last_unstale":"2024-09-12T03:26:14.685256+0000","last_undegraded":"2024-09-12T03:26:14.685256+0000","last_fullsized":"2024-09-12T03:26:14.685256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:00.423368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684093+0000","last_change":"2024-09-12T03:24:31.646006+0000","last_active":"2024-09-12T03:26:14.684093+0000","last_peered":"2024-09-12T03:26:14.684093+0000","last_clean":"2024-09-12T03:26:14.684093+0000","last_became_active":"2024-09-12T03:24:31.643165+0000","last_became_peered":"2024-09-12T03:24:31.643165+0000","last_unstale":"2024-09-12T03:26:14.684093+0000","last_undegraded":"2024-09-12T03:26:14.684093+0000","last_fullsized":"2024-09-12T03:26:14.684093+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":101,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:18.948220+0000","last_change":"2024-09-12T03:24:29.304521+0000","last_active":"2024-09-12T03:26:18.948220+0000","last_peered":"2024-09-12T03:26:18.948220+0000","last_clean":"2024-09-12T03:26:18.948220+0000","last_became_active":"2024-09-12T03:24:29.304147+0000","last_became_peered":"2024-09-12T03:24:29.304147+0000","last_unstale":"2024-09-12T03:26:18.948220+0000","last_undegraded":"2024-09-12T03:26:18.948220+0000","last_fullsized":"2024-09-12T03:26:18.948220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:17.834282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660722+0000","last_change":"2024-09-12T03:24:40.706844+0000","last_active":"2024-09-12T03:26:14.660722+0000","last_peered":"2024-09-12T03:26:14.660722+0000","last_clean":"2024-09-12T03:26:14.660722+0000","last_became_active":"2024-09-12T03:24:40.704829+0000","last_became_peered":"2024-09-12T03:24:40.704829+0000","last_unstale":"2024-09-12T03:26:14.660722+0000","last_undegraded":"2024-09-12T03:26:14.660722+0000","last_fullsized":"2024-09-12T03:26:14.660722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:32.875316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684487+0000","last_change":"2024-09-12T03:24:31.650168+0000","last_active":"2024-09-12T03:26:14.684487+0000","last_peered":"2024-09-12T03:26:14.684487+0000","last_clean":"2024-09-12T03:26:14.684487+0000","last_became_active":"2024-09-12T03:24:31.649754+0000","last_became_peered":"2024-09-12T03:24:31.649754+0000","last_unstale":"2024-09-12T03:26:14.684487+0000","last_undegraded":"2024-09-12T03:26:14.684487+0000","last_fullsized":"2024-09-12T03:26:14.684487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:09.615836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670514+0000","last_change":"2024-09-12T03:24:29.303488+0000","last_active":"2024-09-12T03:26:14.670514+0000","last_peered":"2024-09-12T03:26:14.670514+0000","last_clean":"2024-09-12T03:26:14.670514+0000","last_became_active":"2024-09-12T03:24:29.301858+0000","last_became_peered":"2024-09-12T03:24:29.301858+0000","last_unstale":"2024-09-12T03:26:14.670514+0000","last_undegraded":"2024-09-12T03:26:14.670514+0000","last_fullsized":"2024-09-12T03:26:14.670514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:27.846820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667760+0000","last_change":"2024-09-12T03:24:40.697734+0000","last_active":"2024-09-12T03:26:14.667760+0000","last_peered":"2024-09-12T03:26:14.667760+0000","last_clean":"2024-09-12T03:26:14.667760+0000","last_became_active":"2024-09-12T03:24:40.697271+0000","last_became_peered":"2024-09-12T03:24:40.697271+0000","last_unstale":"2024-09-12T03:26:14.667760+0000","last_undegraded":"2024-09-12T03:26:14.667760+0000","last_fullsized":"2024-09-12T03:26:14.667760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.336322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684740+0000","last_change":"2024-09-12T03:24:31.647040+0000","last_active":"2024-09-12T03:26:14.684740+0000","last_peered":"2024-09-12T03:26:14.684740+0000","last_clean":"2024-09-12T03:26:14.684740+0000","last_became_active":"2024-09-12T03:24:31.643365+0000","last_became_peered":"2024-09-12T03:24:31.643365+0000","last_unstale":"2024-09-12T03:26:14.684740+0000","last_undegraded":"2024-09-12T03:26:14.684740+0000","last_fullsized":"2024-09-12T03:26:14.684740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:06.638005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663708+0000","last_change":"2024-09-12T03:24:29.303390+0000","last_active":"2024-09-12T03:26:14.663708+0000","last_peered":"2024-09-12T03:26:14.663708+0000","last_clean":"2024-09-12T03:26:14.663708+0000","last_became_active":"2024-09-12T03:24:29.303072+0000","last_became_peered":"2024-09-12T03:24:29.303072+0000","last_unstale":"2024-09-12T03:26:14.663708+0000","last_undegraded":"2024-09-12T03:26:14.663708+0000","last_fullsized":"2024-09-12T03:26:14.663708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:01.572888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670400+0000","last_change":"2024-09-12T03:24:40.707977+0000","last_active":"2024-09-12T03:26:14.670400+0000","last_peered":"2024-09-12T03:26:14.670400+0000","last_clean":"2024-09-12T03:26:14.670400+0000","last_became_active":"2024-09-12T03:24:40.707497+0000","last_became_peered":"2024-09-12T03:24:40.707497+0000","last_unstale":"2024-09-12T03:26:14.670400+0000","last_undegraded":"2024-09-12T03:26:14.670400+0000","last_fullsized":"2024-09-12T03:26:14.670400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:45.474021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665523+0000","last_change":"2024-09-12T03:24:31.642417+0000","last_active":"2024-09-12T03:26:14.665523+0000","last_peered":"2024-09-12T03:26:14.665523+0000","last_clean":"2024-09-12T03:26:14.665523+0000","last_became_active":"2024-09-12T03:24:31.642076+0000","last_became_peered":"2024-09-12T03:24:31.642076+0000","last_unstale":"2024-09-12T03:26:14.665523+0000","last_undegraded":"2024-09-12T03:26:14.665523+0000","last_fullsized":"2024-09-12T03:26:14.665523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:56.436088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:18.954263+0000","last_change":"2024-09-12T03:24:29.302982+0000","last_active":"2024-09-12T03:26:18.954263+0000","last_peered":"2024-09-12T03:26:18.954263+0000","last_clean":"2024-09-12T03:26:18.954263+0000","last_became_active":"2024-09-12T03:24:29.302369+0000","last_became_peered":"2024-09-12T03:24:29.302369+0000","last_unstale":"2024-09-12T03:26:18.954263+0000","last_undegraded":"2024-09-12T03:26:18.954263+0000","last_fullsized":"2024-09-12T03:26:18.954263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:24:30.064170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":124,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660148+0000","last_change":"2024-09-12T03:24:40.696745+0000","last_active":"2024-09-12T03:26:14.660148+0000","last_peered":"2024-09-12T03:26:14.660148+0000","last_clean":"2024-09-12T03:26:14.660148+0000","last_became_active":"2024-09-12T03:24:40.696273+0000","last_became_peered":"2024-09-12T03:24:40.696273+0000","last_unstale":"2024-09-12T03:26:14.660148+0000","last_undegraded":"2024-09-12T03:26:14.660148+0000","last_fullsized":"2024-09-12T03:26:14.660148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:38.519765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670523+0000","last_change":"2024-09-12T03:24:31.650567+0000","last_active":"2024-09-12T03:26:14.670523+0000","last_peered":"2024-09-12T03:26:14.670523+0000","last_clean":"2024-09-12T03:26:14.670523+0000","last_became_active":"2024-09-12T03:24:31.650242+0000","last_became_peered":"2024-09-12T03:24:31.650242+0000","last_unstale":"2024-09-12T03:26:14.670523+0000","last_undegraded":"2024-09-12T03:26:14.670523+0000","last_fullsized":"2024-09-12T03:26:14.670523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:14.799301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670630+0000","last_change":"2024-09-12T03:24:29.310630+0000","last_active":"2024-09-12T03:26:14.670630+0000","last_peered":"2024-09-12T03:26:14.670630+0000","last_clean":"2024-09-12T03:26:14.670630+0000","last_became_active":"2024-09-12T03:24:29.310302+0000","last_became_peered":"2024-09-12T03:24:29.310302+0000","last_unstale":"2024-09-12T03:26:14.670630+0000","last_undegraded":"2024-09-12T03:26:14.670630+0000","last_fullsized":"2024-09-12T03:26:14.670630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:53.835246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669376+0000","last_change":"2024-09-12T03:24:40.698941+0000","last_active":"2024-09-12T03:26:14.669376+0000","last_peered":"2024-09-12T03:26:14.669376+0000","last_clean":"2024-09-12T03:26:14.669376+0000","last_became_active":"2024-09-12T03:24:40.698323+0000","last_became_peered":"2024-09-12T03:24:40.698323+0000","last_unstale":"2024-09-12T03:26:14.669376+0000","last_undegraded":"2024-09-12T03:26:14.669376+0000","last_fullsized":"2024-09-12T03:26:14.669376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:29.425118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669098+0000","last_change":"2024-09-12T03:25:14.320685+0000","last_active":"2024-09-12T03:26:14.669098+0000","last_peered":"2024-09-12T03:26:14.669098+0000","last_clean":"2024-09-12T03:26:14.669098+0000","last_became_active":"2024-09-12T03:25:14.320295+0000","last_became_peered":"2024-09-12T03:25:14.320295+0000","last_unstale":"2024-09-12T03:26:14.669098+0000","last_undegraded":"2024-09-12T03:26:14.669098+0000","last_fullsized":"2024-09-12T03:26:14.669098+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:42.459136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684539+0000","last_change":"2024-09-12T03:24:29.302988+0000","last_active":"2024-09-12T03:26:14.684539+0000","last_peered":"2024-09-12T03:26:14.684539+0000","last_clean":"2024-09-12T03:26:14.684539+0000","last_became_active":"2024-09-12T03:24:29.302569+0000","last_became_peered":"2024-09-12T03:24:29.302569+0000","last_unstale":"2024-09-12T03:26:14.684539+0000","last_undegraded":"2024-09-12T03:26:14.684539+0000","last_fullsized":"2024-09-12T03:26:14.684539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:00.196284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660084+0000","last_change":"2024-09-12T03:24:40.701486+0000","last_active":"2024-09-12T03:26:14.660084+0000","last_peered":"2024-09-12T03:26:14.660084+0000","last_clean":"2024-09-12T03:26:14.660084+0000","last_became_active":"2024-09-12T03:24:40.701078+0000","last_became_peered":"2024-09-12T03:24:40.701078+0000","last_unstale":"2024-09-12T03:26:14.660084+0000","last_undegraded":"2024-09-12T03:26:14.660084+0000","last_fullsized":"2024-09-12T03:26:14.660084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:50.032773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670112+0000","last_change":"2024-09-12T03:24:31.646681+0000","last_active":"2024-09-12T03:26:14.670112+0000","last_peered":"2024-09-12T03:26:14.670112+0000","last_clean":"2024-09-12T03:26:14.670112+0000","last_became_active":"2024-09-12T03:24:31.646389+0000","last_became_peered":"2024-09-12T03:24:31.646389+0000","last_unstale":"2024-09-12T03:26:14.670112+0000","last_undegraded":"2024-09-12T03:26:14.670112+0000","last_fullsized":"2024-09-12T03:26:14.670112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:37.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":98,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:17.946602+0000","last_change":"2024-09-12T03:24:29.299316+0000","last_active":"2024-09-12T03:26:17.946602+0000","last_peered":"2024-09-12T03:26:17.946602+0000","last_clean":"2024-09-12T03:26:17.946602+0000","last_became_active":"2024-09-12T03:24:29.298900+0000","last_became_peered":"2024-09-12T03:24:29.298900+0000","last_unstale":"2024-09-12T03:26:17.946602+0000","last_undegraded":"2024-09-12T03:26:17.946602+0000","last_fullsized":"2024-09-12T03:26:17.946602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:36.650938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684874+0000","last_change":"2024-09-12T03:24:40.702034+0000","last_active":"2024-09-12T03:26:14.684874+0000","last_peered":"2024-09-12T03:26:14.684874+0000","last_clean":"2024-09-12T03:26:14.684874+0000","last_became_active":"2024-09-12T03:24:40.699312+0000","last_became_peered":"2024-09-12T03:24:40.699312+0000","last_unstale":"2024-09-12T03:26:14.684874+0000","last_undegraded":"2024-09-12T03:26:14.684874+0000","last_fullsized":"2024-09-12T03:26:14.684874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:58.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668712+0000","last_change":"2024-09-12T03:24:40.701744+0000","last_active":"2024-09-12T03:26:14.668712+0000","last_peered":"2024-09-12T03:26:14.668712+0000","last_clean":"2024-09-12T03:26:14.668712+0000","last_became_active":"2024-09-12T03:24:40.700929+0000","last_became_peered":"2024-09-12T03:24:40.700929+0000","last_unstale":"2024-09-12T03:26:14.668712+0000","last_undegraded":"2024-09-12T03:26:14.668712+0000","last_fullsized":"2024-09-12T03:26:14.668712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:05.851810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660177+0000","last_change":"2024-09-12T03:24:31.647774+0000","last_active":"2024-09-12T03:26:14.660177+0000","last_peered":"2024-09-12T03:26:14.660177+0000","last_clean":"2024-09-12T03:26:14.660177+0000","last_became_active":"2024-09-12T03:24:31.647329+0000","last_became_peered":"2024-09-12T03:24:31.647329+0000","last_unstale":"2024-09-12T03:26:14.660177+0000","last_undegraded":"2024-09-12T03:26:14.660177+0000","last_fullsized":"2024-09-12T03:26:14.660177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:00.868358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670540+0000","last_change":"2024-09-12T03:24:29.307455+0000","last_active":"2024-09-12T03:26:14.670540+0000","last_peered":"2024-09-12T03:26:14.670540+0000","last_clean":"2024-09-12T03:26:14.670540+0000","last_became_active":"2024-09-12T03:24:29.307120+0000","last_became_peered":"2024-09-12T03:24:29.307120+0000","last_unstale":"2024-09-12T03:26:14.670540+0000","last_undegraded":"2024-09-12T03:26:14.670540+0000","last_fullsized":"2024-09-12T03:26:14.670540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:29.374097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667548+0000","last_change":"2024-09-12T03:24:40.695149+0000","last_active":"2024-09-12T03:26:14.667548+0000","last_peered":"2024-09-12T03:26:14.667548+0000","last_clean":"2024-09-12T03:26:14.667548+0000","last_became_active":"2024-09-12T03:24:40.694755+0000","last_became_peered":"2024-09-12T03:24:40.694755+0000","last_unstale":"2024-09-12T03:26:14.667548+0000","last_undegraded":"2024-09-12T03:26:14.667548+0000","last_fullsized":"2024-09-12T03:26:14.667548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:34.045481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684978+0000","last_change":"2024-09-12T03:24:31.654674+0000","last_active":"2024-09-12T03:26:14.684978+0000","last_peered":"2024-09-12T03:26:14.684978+0000","last_clean":"2024-09-12T03:26:14.684978+0000","last_became_active":"2024-09-12T03:24:31.654319+0000","last_became_peered":"2024-09-12T03:24:31.654319+0000","last_unstale":"2024-09-12T03:26:14.684978+0000","last_undegraded":"2024-09-12T03:26:14.684978+0000","last_fullsized":"2024-09-12T03:26:14.684978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:33.441162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684538+0000","last_change":"2024-09-12T03:24:29.307849+0000","last_active":"2024-09-12T03:26:14.684538+0000","last_peered":"2024-09-12T03:26:14.684538+0000","last_clean":"2024-09-12T03:26:14.684538+0000","last_became_active":"2024-09-12T03:24:29.306017+0000","last_became_peered":"2024-09-12T03:24:29.306017+0000","last_unstale":"2024-09-12T03:26:14.684538+0000","last_undegraded":"2024-09-12T03:26:14.684538+0000","last_fullsized":"2024-09-12T03:26:14.684538+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:37.478289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663950+0000","last_change":"2024-09-12T03:24:29.302410+0000","last_active":"2024-09-12T03:26:14.663950+0000","last_peered":"2024-09-12T03:26:14.663950+0000","last_clean":"2024-09-12T03:26:14.663950+0000","last_became_active":"2024-09-12T03:24:29.302031+0000","last_became_peered":"2024-09-12T03:24:29.302031+0000","last_unstale":"2024-09-12T03:26:14.663950+0000","last_undegraded":"2024-09-12T03:26:14.663950+0000","last_fullsized":"2024-09-12T03:26:14.663950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:11.698690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670272+0000","last_change":"2024-09-12T03:24:31.652823+0000","last_active":"2024-09-12T03:26:14.670272+0000","last_peered":"2024-09-12T03:26:14.670272+0000","last_clean":"2024-09-12T03:26:14.670272+0000","last_became_active":"2024-09-12T03:24:31.652438+0000","last_became_peered":"2024-09-12T03:24:31.652438+0000","last_unstale":"2024-09-12T03:26:14.670272+0000","last_undegraded":"2024-09-12T03:26:14.670272+0000","last_fullsized":"2024-09-12T03:26:14.670272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:28.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663716+0000","last_change":"2024-09-12T03:25:13.942339+0000","last_active":"2024-09-12T03:26:14.663716+0000","last_peered":"2024-09-12T03:26:14.663716+0000","last_clean":"2024-09-12T03:26:14.663716+0000","last_became_active":"2024-09-12T03:25:13.942030+0000","last_became_peered":"2024-09-12T03:25:13.942030+0000","last_unstale":"2024-09-12T03:26:14.663716+0000","last_undegraded":"2024-09-12T03:26:14.663716+0000","last_fullsized":"2024-09-12T03:26:14.663716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:39.584996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,10],"acting":[2,6,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:17.947215+0000","last_change":"2024-09-12T03:24:29.303318+0000","last_active":"2024-09-12T03:26:17.947215+0000","last_peered":"2024-09-12T03:26:17.947215+0000","last_clean":"2024-09-12T03:26:17.947215+0000","last_became_active":"2024-09-12T03:24:29.303001+0000","last_became_peered":"2024-09-12T03:24:29.303001+0000","last_unstale":"2024-09-12T03:26:17.947215+0000","last_undegraded":"2024-09-12T03:26:17.947215+0000","last_fullsized":"2024-09-12T03:26:17.947215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:28.287523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:28.287523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:08.094697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665180+0000","last_change":"2024-09-12T03:24:31.644465+0000","last_active":"2024-09-12T03:26:14.665180+0000","last_peered":"2024-09-12T03:26:14.665180+0000","last_clean":"2024-09-12T03:26:14.665180+0000","last_became_active":"2024-09-12T03:24:31.644141+0000","last_became_peered":"2024-09-12T03:24:31.644141+0000","last_unstale":"2024-09-12T03:26:14.665180+0000","last_undegraded":"2024-09-12T03:26:14.665180+0000","last_fullsized":"2024-09-12T03:26:14.665180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:37.448614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663896+0000","last_change":"2024-09-12T03:24:40.695190+0000","last_active":"2024-09-12T03:26:14.663896+0000","last_peered":"2024-09-12T03:26:14.663896+0000","last_clean":"2024-09-12T03:26:14.663896+0000","last_became_active":"2024-09-12T03:24:40.694746+0000","last_became_peered":"2024-09-12T03:24:40.694746+0000","last_unstale":"2024-09-12T03:26:14.663896+0000","last_undegraded":"2024-09-12T03:26:14.663896+0000","last_fullsized":"2024-09-12T03:26:14.663896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:39.669885+0000","last_clean_scrub_stamp":"2024-09-12T03:24:39.669885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:31.905437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667099+0000","last_change":"2024-09-12T03:24:31.649068+0000","last_active":"2024-09-12T03:26:14.667099+0000","last_peered":"2024-09-12T03:26:14.667099+0000","last_clean":"2024-09-12T03:26:14.667099+0000","last_became_active":"2024-09-12T03:24:31.648761+0000","last_became_peered":"2024-09-12T03:24:31.648761+0000","last_unstale":"2024-09-12T03:26:14.667099+0000","last_undegraded":"2024-09-12T03:26:14.667099+0000","last_fullsized":"2024-09-12T03:26:14.667099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:08.818521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663592+0000","last_change":"2024-09-12T03:24:31.643604+0000","last_active":"2024-09-12T03:26:14.663592+0000","last_peered":"2024-09-12T03:26:14.663592+0000","last_clean":"2024-09-12T03:26:14.663592+0000","last_became_active":"2024-09-12T03:24:31.643053+0000","last_became_peered":"2024-09-12T03:24:31.643053+0000","last_unstale":"2024-09-12T03:26:14.663592+0000","last_undegraded":"2024-09-12T03:26:14.663592+0000","last_fullsized":"2024-09-12T03:26:14.663592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:59.814891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663349+0000","last_change":"2024-09-12T03:24:31.648376+0000","last_active":"2024-09-12T03:26:14.663349+0000","last_peered":"2024-09-12T03:26:14.663349+0000","last_clean":"2024-09-12T03:26:14.663349+0000","last_became_active":"2024-09-12T03:24:31.648076+0000","last_became_peered":"2024-09-12T03:24:31.648076+0000","last_unstale":"2024-09-12T03:26:14.663349+0000","last_undegraded":"2024-09-12T03:26:14.663349+0000","last_fullsized":"2024-09-12T03:26:14.663349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:47.736362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668249+0000","last_change":"2024-09-12T03:24:31.652489+0000","last_active":"2024-09-12T03:26:14.668249+0000","last_peered":"2024-09-12T03:26:14.668249+0000","last_clean":"2024-09-12T03:26:14.668249+0000","last_became_active":"2024-09-12T03:24:31.651907+0000","last_became_peered":"2024-09-12T03:24:31.651907+0000","last_unstale":"2024-09-12T03:26:14.668249+0000","last_undegraded":"2024-09-12T03:26:14.668249+0000","last_fullsized":"2024-09-12T03:26:14.668249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:24.584264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684936+0000","last_change":"2024-09-12T03:24:31.649967+0000","last_active":"2024-09-12T03:26:14.684936+0000","last_peered":"2024-09-12T03:26:14.684936+0000","last_clean":"2024-09-12T03:26:14.684936+0000","last_became_active":"2024-09-12T03:24:31.649687+0000","last_became_peered":"2024-09-12T03:24:31.649687+0000","last_unstale":"2024-09-12T03:26:14.684936+0000","last_undegraded":"2024-09-12T03:26:14.684936+0000","last_fullsized":"2024-09-12T03:26:14.684936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:14.875352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667562+0000","last_change":"2024-09-12T03:24:31.645115+0000","last_active":"2024-09-12T03:26:14.667562+0000","last_peered":"2024-09-12T03:26:14.667562+0000","last_clean":"2024-09-12T03:26:14.667562+0000","last_became_active":"2024-09-12T03:24:31.644442+0000","last_became_peered":"2024-09-12T03:24:31.644442+0000","last_unstale":"2024-09-12T03:26:14.667562+0000","last_undegraded":"2024-09-12T03:26:14.667562+0000","last_fullsized":"2024-09-12T03:26:14.667562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:28.539948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.664091+0000","last_change":"2024-09-12T03:24:31.643283+0000","last_active":"2024-09-12T03:26:14.664091+0000","last_peered":"2024-09-12T03:26:14.664091+0000","last_clean":"2024-09-12T03:26:14.664091+0000","last_became_active":"2024-09-12T03:24:31.642935+0000","last_became_peered":"2024-09-12T03:24:31.642935+0000","last_unstale":"2024-09-12T03:26:14.664091+0000","last_undegraded":"2024-09-12T03:26:14.664091+0000","last_fullsized":"2024-09-12T03:26:14.664091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:30.754875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660623+0000","last_change":"2024-09-12T03:25:13.817610+0000","last_active":"2024-09-12T03:26:14.660623+0000","last_peered":"2024-09-12T03:26:14.660623+0000","last_clean":"2024-09-12T03:26:14.660623+0000","last_became_active":"2024-09-12T03:25:13.817008+0000","last_became_peered":"2024-09-12T03:25:13.817008+0000","last_unstale":"2024-09-12T03:26:14.660623+0000","last_undegraded":"2024-09-12T03:26:14.660623+0000","last_fullsized":"2024-09-12T03:26:14.660623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:45.334353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670205+0000","last_change":"2024-09-12T03:24:31.656588+0000","last_active":"2024-09-12T03:26:14.670205+0000","last_peered":"2024-09-12T03:26:14.670205+0000","last_clean":"2024-09-12T03:26:14.670205+0000","last_became_active":"2024-09-12T03:24:31.656294+0000","last_became_peered":"2024-09-12T03:24:31.656294+0000","last_unstale":"2024-09-12T03:26:14.670205+0000","last_undegraded":"2024-09-12T03:26:14.670205+0000","last_fullsized":"2024-09-12T03:26:14.670205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.855802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685017+0000","last_change":"2024-09-12T03:24:31.648986+0000","last_active":"2024-09-12T03:26:14.685017+0000","last_peered":"2024-09-12T03:26:14.685017+0000","last_clean":"2024-09-12T03:26:14.685017+0000","last_became_active":"2024-09-12T03:24:31.648144+0000","last_became_peered":"2024-09-12T03:24:31.648144+0000","last_unstale":"2024-09-12T03:26:14.685017+0000","last_undegraded":"2024-09-12T03:26:14.685017+0000","last_fullsized":"2024-09-12T03:26:14.685017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.823742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670391+0000","last_change":"2024-09-12T03:24:31.644390+0000","last_active":"2024-09-12T03:26:14.670391+0000","last_peered":"2024-09-12T03:26:14.670391+0000","last_clean":"2024-09-12T03:26:14.670391+0000","last_became_active":"2024-09-12T03:24:31.642660+0000","last_became_peered":"2024-09-12T03:24:31.642660+0000","last_unstale":"2024-09-12T03:26:14.670391+0000","last_undegraded":"2024-09-12T03:26:14.670391+0000","last_fullsized":"2024-09-12T03:26:14.670391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:58.577883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685264+0000","last_change":"2024-09-12T03:24:31.649895+0000","last_active":"2024-09-12T03:26:14.685264+0000","last_peered":"2024-09-12T03:26:14.685264+0000","last_clean":"2024-09-12T03:26:14.685264+0000","last_became_active":"2024-09-12T03:24:31.649141+0000","last_became_peered":"2024-09-12T03:24:31.649141+0000","last_unstale":"2024-09-12T03:26:14.685264+0000","last_undegraded":"2024-09-12T03:26:14.685264+0000","last_fullsized":"2024-09-12T03:26:14.685264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:59:48.175496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667879+0000","last_change":"2024-09-12T03:24:31.640402+0000","last_active":"2024-09-12T03:26:14.667879+0000","last_peered":"2024-09-12T03:26:14.667879+0000","last_clean":"2024-09-12T03:26:14.667879+0000","last_became_active":"2024-09-12T03:24:31.639987+0000","last_became_peered":"2024-09-12T03:24:31.639987+0000","last_unstale":"2024-09-12T03:26:14.667879+0000","last_undegraded":"2024-09-12T03:26:14.667879+0000","last_fullsized":"2024-09-12T03:26:14.667879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:51.540149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.664073+0000","last_change":"2024-09-12T03:24:31.647524+0000","last_active":"2024-09-12T03:26:14.664073+0000","last_peered":"2024-09-12T03:26:14.664073+0000","last_clean":"2024-09-12T03:26:14.664073+0000","last_became_active":"2024-09-12T03:24:31.647204+0000","last_became_peered":"2024-09-12T03:24:31.647204+0000","last_unstale":"2024-09-12T03:26:14.664073+0000","last_undegraded":"2024-09-12T03:26:14.664073+0000","last_fullsized":"2024-09-12T03:26:14.664073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.895586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.660706+0000","last_change":"2024-09-12T03:24:31.640797+0000","last_active":"2024-09-12T03:26:14.660706+0000","last_peered":"2024-09-12T03:26:14.660706+0000","last_clean":"2024-09-12T03:26:14.660706+0000","last_became_active":"2024-09-12T03:24:31.640479+0000","last_became_peered":"2024-09-12T03:24:31.640479+0000","last_unstale":"2024-09-12T03:26:14.660706+0000","last_undegraded":"2024-09-12T03:26:14.660706+0000","last_fullsized":"2024-09-12T03:26:14.660706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:08.762939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667947+0000","last_change":"2024-09-12T03:24:31.644225+0000","last_active":"2024-09-12T03:26:14.667947+0000","last_peered":"2024-09-12T03:26:14.667947+0000","last_clean":"2024-09-12T03:26:14.667947+0000","last_became_active":"2024-09-12T03:24:31.643877+0000","last_became_peered":"2024-09-12T03:24:31.643877+0000","last_unstale":"2024-09-12T03:26:14.667947+0000","last_undegraded":"2024-09-12T03:26:14.667947+0000","last_fullsized":"2024-09-12T03:26:14.667947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:50.511392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669839+0000","last_change":"2024-09-12T03:24:31.644917+0000","last_active":"2024-09-12T03:26:14.669839+0000","last_peered":"2024-09-12T03:26:14.669839+0000","last_clean":"2024-09-12T03:26:14.669839+0000","last_became_active":"2024-09-12T03:24:31.644591+0000","last_became_peered":"2024-09-12T03:24:31.644591+0000","last_unstale":"2024-09-12T03:26:14.669839+0000","last_undegraded":"2024-09-12T03:26:14.669839+0000","last_fullsized":"2024-09-12T03:26:14.669839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:21.961959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668067+0000","last_change":"2024-09-12T03:24:31.648146+0000","last_active":"2024-09-12T03:26:14.668067+0000","last_peered":"2024-09-12T03:26:14.668067+0000","last_clean":"2024-09-12T03:26:14.668067+0000","last_became_active":"2024-09-12T03:24:31.641423+0000","last_became_peered":"2024-09-12T03:24:31.641423+0000","last_unstale":"2024-09-12T03:26:14.668067+0000","last_undegraded":"2024-09-12T03:26:14.668067+0000","last_fullsized":"2024-09-12T03:26:14.668067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:50.985126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668440+0000","last_change":"2024-09-12T03:24:31.641030+0000","last_active":"2024-09-12T03:26:14.668440+0000","last_peered":"2024-09-12T03:26:14.668440+0000","last_clean":"2024-09-12T03:26:14.668440+0000","last_became_active":"2024-09-12T03:24:31.639015+0000","last_became_peered":"2024-09-12T03:24:31.639015+0000","last_unstale":"2024-09-12T03:26:14.668440+0000","last_undegraded":"2024-09-12T03:26:14.668440+0000","last_fullsized":"2024-09-12T03:26:14.668440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.961304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.685232+0000","last_change":"2024-09-12T03:24:31.644601+0000","last_active":"2024-09-12T03:26:14.685232+0000","last_peered":"2024-09-12T03:26:14.685232+0000","last_clean":"2024-09-12T03:26:14.685232+0000","last_became_active":"2024-09-12T03:24:31.644122+0000","last_became_peered":"2024-09-12T03:24:31.644122+0000","last_unstale":"2024-09-12T03:26:14.685232+0000","last_undegraded":"2024-09-12T03:26:14.685232+0000","last_fullsized":"2024-09-12T03:26:14.685232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:06.335464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670199+0000","last_change":"2024-09-12T03:24:31.641496+0000","last_active":"2024-09-12T03:26:14.670199+0000","last_peered":"2024-09-12T03:26:14.670199+0000","last_clean":"2024-09-12T03:26:14.670199+0000","last_became_active":"2024-09-12T03:24:31.641195+0000","last_became_peered":"2024-09-12T03:24:31.641195+0000","last_unstale":"2024-09-12T03:26:14.670199+0000","last_undegraded":"2024-09-12T03:26:14.670199+0000","last_fullsized":"2024-09-12T03:26:14.670199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:02.882398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684560+0000","last_change":"2024-09-12T03:24:31.645766+0000","last_active":"2024-09-12T03:26:14.684560+0000","last_peered":"2024-09-12T03:26:14.684560+0000","last_clean":"2024-09-12T03:26:14.684560+0000","last_became_active":"2024-09-12T03:24:31.644903+0000","last_became_peered":"2024-09-12T03:24:31.644903+0000","last_unstale":"2024-09-12T03:26:14.684560+0000","last_undegraded":"2024-09-12T03:26:14.684560+0000","last_fullsized":"2024-09-12T03:26:14.684560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.664128+0000","last_change":"2024-09-12T03:25:13.813333+0000","last_active":"2024-09-12T03:26:14.664128+0000","last_peered":"2024-09-12T03:26:14.664128+0000","last_clean":"2024-09-12T03:26:14.664128+0000","last_became_active":"2024-09-12T03:25:13.812963+0000","last_became_peered":"2024-09-12T03:25:13.812963+0000","last_unstale":"2024-09-12T03:26:14.664128+0000","last_undegraded":"2024-09-12T03:26:14.664128+0000","last_fullsized":"2024-09-12T03:26:14.664128+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:38.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.670223+0000","last_change":"2024-09-12T03:25:14.266892+0000","last_active":"2024-09-12T03:26:14.670223+0000","last_peered":"2024-09-12T03:26:14.670223+0000","last_clean":"2024-09-12T03:26:14.670223+0000","last_became_active":"2024-09-12T03:25:14.266452+0000","last_became_peered":"2024-09-12T03:25:14.266452+0000","last_unstale":"2024-09-12T03:26:14.670223+0000","last_undegraded":"2024-09-12T03:26:14.670223+0000","last_fullsized":"2024-09-12T03:26:14.670223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:58.206639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684877+0000","last_change":"2024-09-12T03:24:31.646199+0000","last_active":"2024-09-12T03:26:14.684877+0000","last_peered":"2024-09-12T03:26:14.684877+0000","last_clean":"2024-09-12T03:26:14.684877+0000","last_became_active":"2024-09-12T03:24:31.645872+0000","last_became_peered":"2024-09-12T03:24:31.645872+0000","last_unstale":"2024-09-12T03:26:14.684877+0000","last_undegraded":"2024-09-12T03:26:14.684877+0000","last_fullsized":"2024-09-12T03:26:14.684877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:21.538730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.684198+0000","last_change":"2024-09-12T03:24:31.643704+0000","last_active":"2024-09-12T03:26:14.684198+0000","last_peered":"2024-09-12T03:26:14.684198+0000","last_clean":"2024-09-12T03:26:14.684198+0000","last_became_active":"2024-09-12T03:24:31.643190+0000","last_became_peered":"2024-09-12T03:24:31.643190+0000","last_unstale":"2024-09-12T03:26:14.684198+0000","last_undegraded":"2024-09-12T03:26:14.684198+0000","last_fullsized":"2024-09-12T03:26:14.684198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:23.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.663836+0000","last_change":"2024-09-12T03:24:31.643282+0000","last_active":"2024-09-12T03:26:14.663836+0000","last_peered":"2024-09-12T03:26:14.663836+0000","last_clean":"2024-09-12T03:26:14.663836+0000","last_became_active":"2024-09-12T03:24:31.642721+0000","last_became_peered":"2024-09-12T03:24:31.642721+0000","last_unstale":"2024-09-12T03:26:14.663836+0000","last_undegraded":"2024-09-12T03:26:14.663836+0000","last_fullsized":"2024-09-12T03:26:14.663836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:52.025426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.665179+0000","last_change":"2024-09-12T03:24:31.650266+0000","last_active":"2024-09-12T03:26:14.665179+0000","last_peered":"2024-09-12T03:26:14.665179+0000","last_clean":"2024-09-12T03:26:14.665179+0000","last_became_active":"2024-09-12T03:24:31.649887+0000","last_became_peered":"2024-09-12T03:24:31.649887+0000","last_unstale":"2024-09-12T03:26:14.665179+0000","last_undegraded":"2024-09-12T03:26:14.665179+0000","last_fullsized":"2024-09-12T03:26:14.665179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:05.043430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.669867+0000","last_change":"2024-09-12T03:24:31.639381+0000","last_active":"2024-09-12T03:26:14.669867+0000","last_peered":"2024-09-12T03:26:14.669867+0000","last_clean":"2024-09-12T03:26:14.669867+0000","last_became_active":"2024-09-12T03:24:31.639047+0000","last_became_peered":"2024-09-12T03:24:31.639047+0000","last_unstale":"2024-09-12T03:26:14.669867+0000","last_undegraded":"2024-09-12T03:26:14.669867+0000","last_fullsized":"2024-09-12T03:26:14.669867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:57.608026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668170+0000","last_change":"2024-09-12T03:24:31.646682+0000","last_active":"2024-09-12T03:26:14.668170+0000","last_peered":"2024-09-12T03:26:14.668170+0000","last_clean":"2024-09-12T03:26:14.668170+0000","last_became_active":"2024-09-12T03:24:31.645065+0000","last_became_peered":"2024-09-12T03:24:31.645065+0000","last_unstale":"2024-09-12T03:26:14.668170+0000","last_undegraded":"2024-09-12T03:26:14.668170+0000","last_fullsized":"2024-09-12T03:26:14.668170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:27.328188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.668654+0000","last_change":"2024-09-12T03:24:31.641532+0000","last_active":"2024-09-12T03:26:14.668654+0000","last_peered":"2024-09-12T03:26:14.668654+0000","last_clean":"2024-09-12T03:26:14.668654+0000","last_became_active":"2024-09-12T03:24:31.641164+0000","last_became_peered":"2024-09-12T03:24:31.641164+0000","last_unstale":"2024-09-12T03:26:14.668654+0000","last_undegraded":"2024-09-12T03:26:14.668654+0000","last_fullsized":"2024-09-12T03:26:14.668654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:28.984409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:26:14.667931+0000","last_change":"2024-09-12T03:24:31.646372+0000","last_active":"2024-09-12T03:26:14.667931+0000","last_peered":"2024-09-12T03:26:14.667931+0000","last_clean":"2024-09-12T03:26:14.667931+0000","last_became_active":"2024-09-12T03:24:31.644636+0000","last_became_peered":"2024-09-12T03:24:31.644636+0000","last_unstale":"2024-09-12T03:26:14.667931+0000","last_undegraded":"2024-09-12T03:26:14.667931+0000","last_fullsized":"2024-09-12T03:26:14.667931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:24:30.609523+0000","last_clean_scrub_stamp":"2024-09-12T03:24:30.609523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:51.023245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":368,"num_read_kb":254,"num_write":169,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481828,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1081344,"data_stored":538749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1769472,"data_stored":1190059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906989,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":543059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1089536,"data_stored":538861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1687552,"data_stored":1134794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1073152,"data_stored":534139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1695744,"data_stored":1130382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1089536,"data_stored":539371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":534333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1155072,"data_stored":599162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1150976,"data_stored":598225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":535527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:26:24.673 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:26:24.674 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:26:24.674 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:26:24.674 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T03:26:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:24 smithi050 ceph-mon[30378]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 14 B/s, 1 objects/s recovering 2024-09-12T03:26:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:24 smithi142 ceph-mon[35329]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 14 B/s, 1 objects/s recovering 2024-09-12T03:26:25.056 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:24 smithi082 ceph-mon[35046]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 14 B/s, 1 objects/s recovering 2024-09-12T03:26:25.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 -- 172.21.15.50:0/413331773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784074eb0 msgr2=0x7f2784112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 --2- 172.21.15.50:0/413331773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784074eb0 0x7f2784112650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f277002a480 comp rx=0 tx=0).stop 2024-09-12T03:26:25.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 -- 172.21.15.50:0/413331773 shutdown_connections 2024-09-12T03:26:25.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 --2- 172.21.15.50:0/413331773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784112b90 0x7f2784114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 --2- 172.21.15.50:0/413331773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784074eb0 0x7f2784112650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 --2- 172.21.15.50:0/413331773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2784074590 0x7f2784074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 -- 172.21.15.50:0/413331773 >> 172.21.15.50:0/413331773 conn(0x7f27840fd8c0 msgr2=0x7f27840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:25.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 -- 172.21.15.50:0/413331773 shutdown_connections 2024-09-12T03:26:25.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.584+0000 7f278ba00640 1 -- 172.21.15.50:0/413331773 wait complete. 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.585+0000 7f278ba00640 1 Processor -- start 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.585+0000 7f278ba00640 1 -- start start 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.585+0000 7f278ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784074590 0x7f27841a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.585+0000 7f278ba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2784074eb0 0x7f27841a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.585+0000 7f278ba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 0x7f278419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f278ba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2784118540 con 0x7f2784112b90 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f278ba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f27841183c0 con 0x7f2784074590 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f278ba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f27841186c0 con 0x7f2784074eb0 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 0x7f278419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 0x7f278419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49802/0 (socket says 172.21.15.50:49802) 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789a00640 1 -- 172.21.15.50:0/4246600400 learned_addr learned my addr 172.21.15.50:0/4246600400 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:25.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2783e00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2784074eb0 0x7f27841a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789000640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784074590 0x7f27841a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789000640 1 -- 172.21.15.50:0/4246600400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2784074eb0 msgr2=0x7f27841a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789000640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2784074eb0 0x7f27841a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789000640 1 -- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 msgr2=0x7f278419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789000640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 0x7f278419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789000640 1 -- 172.21.15.50:0/4246600400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278419f000 con 0x7f2784074590 2024-09-12T03:26:25.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.586+0000 7f2789a00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 0x7f278419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.587+0000 7f2789000640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784074590 0x7f27841a4710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f277800e750 tx=0x7f277800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.587+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800ccc0 con 0x7f2784074590 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.587+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f277800ee60 con 0x7f2784074590 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.587+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278419f2f0 con 0x7f2784074590 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.587+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778004a00 con 0x7f2784074590 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.587+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841ab5e0 con 0x7f2784074590 2024-09-12T03:26:25.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.588+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2784074970 con 0x7f2784074590 2024-09-12T03:26:25.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.590+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2778004ba0 con 0x7f2784074590 2024-09-12T03:26:25.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.591+0000 7f2781600640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f276c0a2c90 0x7f276c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.592+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f27780cf550 con 0x7f2784074590 2024-09-12T03:26:25.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.592+0000 7f2783e00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f276c0a2c90 0x7f276c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.592+0000 7f2783e00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f276c0a2c90 0x7f276c0a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2770009230 tx=0x7f27700058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:25.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.594+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f277809bfd0 con 0x7f2784074590 2024-09-12T03:26:25.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.820+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f278419fe00 con 0x7f2784074590 2024-09-12T03:26:25.821 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:25 smithi050 ceph-mon[30378]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:26:25.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.820+0000 7f2781600640 1 -- 172.21.15.50:0/4246600400 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f277809b8d0 con 0x7f2784074590 2024-09-12T03:26:25.822 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:25.822 INFO:teuthology.orchestra.run.smithi050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:26:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f276c0a2c90 msgr2=0x7f276c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f276c0a2c90 0x7f276c0a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2770009230 tx=0x7f27700058d0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784074590 msgr2=0x7f27841a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784074590 0x7f27841a4710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f277800e750 tx=0x7f277800ec10 comp rx=0 tx=0).stop 2024-09-12T03:26:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 shutdown_connections 2024-09-12T03:26:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f276c0a2c90 0x7f276c0a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2784112b90 0x7f278419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2784074eb0 0x7f27841a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 --2- 172.21.15.50:0/4246600400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2784074590 0x7f27841a4710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 >> 172.21.15.50:0/4246600400 conn(0x7f27840fd8c0 msgr2=0x7f27840ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 shutdown_connections 2024-09-12T03:26:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:25.823+0000 7f278ba00640 1 -- 172.21.15.50:0/4246600400 wait complete. 2024-09-12T03:26:26.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:25 smithi142 ceph-mon[35329]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:26:26.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:25 smithi082 ceph-mon[35046]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:26:26.560 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:26:26.560 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:26.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:26 smithi050 ceph-mon[30378]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-09-12T03:26:26.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:26 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4246600400' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:26:26.937 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:27.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:26 smithi142 ceph-mon[35329]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-09-12T03:26:27.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:26 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4246600400' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:26:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:26 smithi082 ceph-mon[35046]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 13 B/s, 1 objects/s recovering 2024-09-12T03:26:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:26 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4246600400' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:26:27.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 -- 172.21.15.50:0/1997291687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb4074f40 msgr2=0x7facb410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 --2- 172.21.15.50:0/1997291687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb4074f40 0x7facb410de80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faca0009900 tx=0x7faca002a480 comp rx=0 tx=0).stop 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 -- 172.21.15.50:0/1997291687 shutdown_connections 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 --2- 172.21.15.50:0/1997291687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb410e3c0 0x7facb4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 --2- 172.21.15.50:0/1997291687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb4074f40 0x7facb410de80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 --2- 172.21.15.50:0/1997291687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 -- 172.21.15.50:0/1997291687 >> 172.21.15.50:0/1997291687 conn(0x7facb40fd8c0 msgr2=0x7facb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 -- 172.21.15.50:0/1997291687 shutdown_connections 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.435+0000 7facbb200640 1 -- 172.21.15.50:0/1997291687 wait complete. 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.436+0000 7facbb200640 1 Processor -- start 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.436+0000 7facbb200640 1 -- start start 2024-09-12T03:26:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facbb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facbb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb4074f40 0x7facb419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facbb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb410e3c0 0x7facb419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facbb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7facb4114190 con 0x7facb410e3c0 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facbb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7facb4114010 con 0x7facb4074620 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facbb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7facb4114310 con 0x7facb4074f40 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb410e3c0 0x7facb419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41782/0 (socket says 172.21.15.50:41782) 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 -- 172.21.15.50:0/872480725 learned_addr learned my addr 172.21.15.50:0/872480725 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3400640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb4074f40 0x7facb419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 -- 172.21.15.50:0/872480725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb4074f40 msgr2=0x7facb419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb4074f40 0x7facb419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 -- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb410e3c0 msgr2=0x7facb419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb410e3c0 0x7facb419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.437+0000 7facb3e00640 1 -- 172.21.15.50:0/872480725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb419f620 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.438+0000 7facb3e00640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb419a670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faca400e9f0 tx=0x7faca400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.438+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca400cd80 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.438+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb419f910 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.438+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca4010910 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.438+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4018910 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.438+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a71f0 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.439+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4074a00 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.441+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faca4010ac0 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.442+0000 7facb0c00640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac980a2c10 0x7fac980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.443+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7faca40cfe20 con 0x7facb4074620 2024-09-12T03:26:27.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.443+0000 7facb3400640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac980a2c10 0x7fac980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:27.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.443+0000 7facb3400640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac980a2c10 0x7fac980a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faca0000c00 tx=0x7faca0005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:27.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.444+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faca409c1c0 con 0x7facb4074620 2024-09-12T03:26:27.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.671+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facb4109570 con 0x7facb4074620 2024-09-12T03:26:27.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.672+0000 7facb0c00640 1 -- 172.21.15.50:0/872480725 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7faca40a04c0 con 0x7facb4074620 2024-09-12T03:26:27.673 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:27.673 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"btime":"2024-09-12T03:25:24:859723+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:25:24.725697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24508,24526,24536]},"id":1}]} 2024-09-12T03:26:27.673 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:26:27.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.675+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac980a2c10 msgr2=0x7fac980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:27.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.675+0000 7facbb200640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac980a2c10 0x7fac980a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faca0000c00 tx=0x7faca0005b50 comp rx=0 tx=0).stop 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.675+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 msgr2=0x7facb419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.675+0000 7facbb200640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb419a670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faca400e9f0 tx=0x7faca400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 shutdown_connections 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fac980a2c10 0x7fac980a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7facb410e3c0 0x7facb419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facb4074f40 0x7facb419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 --2- 172.21.15.50:0/872480725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facb4074620 0x7facb419a670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 >> 172.21.15.50:0/872480725 conn(0x7facb40fd8c0 msgr2=0x7facb40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 shutdown_connections 2024-09-12T03:26:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:27.676+0000 7facbb200640 1 -- 172.21.15.50:0/872480725 wait complete. 2024-09-12T03:26:28.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:27 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/872480725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:28.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:27 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/872480725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:27 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/872480725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:28.378 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:28.756 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:29.030 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:28 smithi050 ceph-mon[30378]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:26:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:28 smithi142 ceph-mon[35329]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:26:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:28 smithi082 ceph-mon[35046]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:26:29.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.286+0000 7f502a400640 1 -- 172.21.15.50:0/948218162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5024103ff0 msgr2=0x7f50241043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.286+0000 7f502a400640 1 --2- 172.21.15.50:0/948218162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5024103ff0 0x7f50241043d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5014009900 tx=0x7f501402a410 comp rx=0 tx=0).stop 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 -- 172.21.15.50:0/948218162 shutdown_connections 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 --2- 172.21.15.50:0/948218162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5024109120 0x7f5024110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 --2- 172.21.15.50:0/948218162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50241049a0 0x7f50241089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 --2- 172.21.15.50:0/948218162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5024103ff0 0x7f50241043d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 -- 172.21.15.50:0/948218162 >> 172.21.15.50:0/948218162 conn(0x7f50240ffa90 msgr2=0x7f5024101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 -- 172.21.15.50:0/948218162 shutdown_connections 2024-09-12T03:26:29.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.287+0000 7f502a400640 1 -- 172.21.15.50:0/948218162 wait complete. 2024-09-12T03:26:29.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 Processor -- start 2024-09-12T03:26:29.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 -- start start 2024-09-12T03:26:29.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5024103ff0 0x7f502419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 0x7f502419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5024109120 0x7f502419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5024114190 con 0x7f50241049a0 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5024114010 con 0x7f5024109120 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.288+0000 7f502a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5024114310 con 0x7f5024103ff0 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 0x7f502419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 0x7f502419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36496/0 (socket says 172.21.15.50:36496) 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 -- 172.21.15.50:0/543611844 learned_addr learned my addr 172.21.15.50:0/543611844 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f5023400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5024103ff0 0x7f502419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f5023e00640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5024109120 0x7f502419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 -- 172.21.15.50:0/543611844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5024103ff0 msgr2=0x7f502419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5024103ff0 0x7f502419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 -- 172.21.15.50:0/543611844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5024109120 msgr2=0x7f502419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5024109120 0x7f502419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 -- 172.21.15.50:0/543611844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502419f640 con 0x7f50241049a0 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.289+0000 7f501be00640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 0x7f502419abd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e9f0 tx=0x7f500c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:29.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.290+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00cd80 con 0x7f50241049a0 2024-09-12T03:26:29.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.290+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c005500 con 0x7f50241049a0 2024-09-12T03:26:29.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.290+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502419f930 con 0x7f50241049a0 2024-09-12T03:26:29.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.290+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c010690 con 0x7f50241049a0 2024-09-12T03:26:29.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.290+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a7210 con 0x7f50241049a0 2024-09-12T03:26:29.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.291+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50241043d0 con 0x7f50241049a0 2024-09-12T03:26:29.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.294+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f500c0046c0 con 0x7f50241049a0 2024-09-12T03:26:29.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.295+0000 7f5020c00640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f50080a2c10 0x7f50080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.295+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f500c0cf9c0 con 0x7f50241049a0 2024-09-12T03:26:29.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.295+0000 7f5023400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f50080a2c10 0x7f50080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.296+0000 7f5023400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f50080a2c10 0x7f50080a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5014000c00 tx=0x7f5014005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:29.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.299+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f500c09c3b0 con 0x7f50241049a0 2024-09-12T03:26:29.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.522+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f502410bd70 con 0x7f50241049a0 2024-09-12T03:26:29.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.523+0000 7f5020c00640 1 -- 172.21.15.50:0/543611844 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f500c09bcb0 con 0x7f50241049a0 2024-09-12T03:26:29.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:29.524 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"btime":"2024-09-12T03:25:24:859723+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:25:24.725697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24508,24526,24536]},"id":1}]} 2024-09-12T03:26:29.525 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:26:29.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f50080a2c10 msgr2=0x7f50080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f50080a2c10 0x7f50080a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5014000c00 tx=0x7f5014005b50 comp rx=0 tx=0).stop 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 msgr2=0x7f502419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 0x7f502419abd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e9f0 tx=0x7f500c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 shutdown_connections 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f50080a2c10 0x7f50080a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5024109120 0x7f502419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50241049a0 0x7f502419abd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 --2- 172.21.15.50:0/543611844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5024103ff0 0x7f502419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 >> 172.21.15.50:0/543611844 conn(0x7f50240ffa90 msgr2=0x7f5024100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 shutdown_connections 2024-09-12T03:26:29.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:29.526+0000 7f502a400640 1 -- 172.21.15.50:0/543611844 wait complete. 2024-09-12T03:26:29.846 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:29 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/543611844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:30.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:29 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/543611844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:29 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/543611844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:30.236 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:26:30.610 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:30.969 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:30 smithi050 ceph-mon[30378]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:26:30.969 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:30 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:30 smithi142 ceph-mon[35329]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:26:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:30 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:30 smithi082 ceph-mon[35046]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:26:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:30 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 -- 172.21.15.50:0/666925265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 msgr2=0x7fc2f0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/666925265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 0x7fc2f0110810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009ca0 tx=0x7fc2e802a950 comp rx=0 tx=0).stop 2024-09-12T03:26:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 -- 172.21.15.50:0/666925265 shutdown_connections 2024-09-12T03:26:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/666925265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 0x7fc2f0110810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/666925265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/666925265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 0x7fc2f0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 -- 172.21.15.50:0/666925265 >> 172.21.15.50:0/666925265 conn(0x7fc2f00fd8c0 msgr2=0x7fc2f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:31.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.204+0000 7fc2f5c00640 1 -- 172.21.15.50:0/666925265 shutdown_connections 2024-09-12T03:26:31.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.205+0000 7fc2f5c00640 1 -- 172.21.15.50:0/666925265 wait complete. 2024-09-12T03:26:31.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 Processor -- start 2024-09-12T03:26:31.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 -- start start 2024-09-12T03:26:31.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 0x7fc2f019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:31.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:31.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 0x7fc2f019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:31.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01141c0 con 0x7fc2f010e3c0 2024-09-12T03:26:31.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0114040 con 0x7fc2f0074620 2024-09-12T03:26:31.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2f5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0114340 con 0x7fc2f0074f40 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.206+0000 7fc2ee000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2eea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 0x7fc2f019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45718/0 (socket says 172.21.15.50:45718) 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 -- 172.21.15.50:0/4262170061 learned_addr learned my addr 172.21.15.50:0/4262170061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ef400640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 0x7fc2f019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 -- 172.21.15.50:0/4262170061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 msgr2=0x7fc2f019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:31.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 0x7fc2f019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 -- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 msgr2=0x7fc2f019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 0x7fc2f019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 -- 172.21.15.50:0/4262170061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f019f610 con 0x7fc2f0074f40 2024-09-12T03:26:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2eea00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 0x7fc2f019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.207+0000 7fc2ee000640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f019aba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e9f0 tx=0x7fc2dc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:31.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.208+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00cd80 con 0x7fc2f0074f40 2024-09-12T03:26:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.208+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f019f900 con 0x7fc2f0074f40 2024-09-12T03:26:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.208+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2dc005500 con 0x7fc2f0074f40 2024-09-12T03:26:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.208+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc010690 con 0x7fc2f0074f40 2024-09-12T03:26:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.208+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01a71e0 con 0x7fc2f0074f40 2024-09-12T03:26:31.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.209+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0074a00 con 0x7fc2f0074f40 2024-09-12T03:26:31.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.211+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc2dc0046c0 con 0x7fc2f0074f40 2024-09-12T03:26:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.213+0000 7fc2e7400640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc2d40a2c10 0x7fc2d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:31.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.213+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc2dc0cf980 con 0x7fc2f0074f40 2024-09-12T03:26:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.213+0000 7fc2eea00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc2d40a2c10 0x7fc2d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.214+0000 7fc2eea00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc2d40a2c10 0x7fc2d40a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00095f0 tx=0x7fc2e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:31.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.217+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2dc09c370 con 0x7fc2f0074f40 2024-09-12T03:26:31.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.420+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2f002d7c0 con 0x7fc2f0074f40 2024-09-12T03:26:31.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.422+0000 7fc2e7400640 1 -- 172.21.15.50:0/4262170061 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24820 (secure 0 0 0) 0x7fc2dc09bc70 con 0x7fc2f0074f40 2024-09-12T03:26:31.423 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:31.423 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":94,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:26:14.656446+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc2d40a2c10 msgr2=0x7fc2d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc2d40a2c10 0x7fc2d40a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00095f0 tx=0x7fc2e0005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 msgr2=0x7fc2f019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f019aba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e9f0 tx=0x7fc2dc00edf0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 shutdown_connections 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc2d40a2c10 0x7fc2d40a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010e3c0 0x7fc2f019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2f0074f40 0x7fc2f019aba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 --2- 172.21.15.50:0/4262170061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2f0074620 0x7fc2f019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 >> 172.21.15.50:0/4262170061 conn(0x7fc2f00fd8c0 msgr2=0x7fc2f00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 shutdown_connections 2024-09-12T03:26:31.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:31.425+0000 7fc2f5c00640 1 -- 172.21.15.50:0/4262170061 wait complete. 2024-09-12T03:26:31.958 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:31 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4262170061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:32.014 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:31 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4262170061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:32.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:31 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4262170061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:32.617 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:32 smithi050 ceph-mon[30378]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:26:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:32.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:32 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.012 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:32 smithi142 ceph-mon[35329]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:26:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:32 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:32 smithi082 ceph-mon[35046]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:26:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:32 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:33.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 -- 172.21.15.50:0/3281333703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4074ee0 msgr2=0x7fc3f410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/3281333703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4074ee0 0x7fc3f410dd20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009860 tx=0x7fc3dc02a3d0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 -- 172.21.15.50:0/3281333703 shutdown_connections 2024-09-12T03:26:33.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/3281333703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3f410e260 0x7fc3f4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/3281333703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4074ee0 0x7fc3f410dd20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/3281333703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f40745c0 0x7fc3f40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 -- 172.21.15.50:0/3281333703 >> 172.21.15.50:0/3281333703 conn(0x7fc3f40fd6e0 msgr2=0x7fc3f40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:33.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 -- 172.21.15.50:0/3281333703 shutdown_connections 2024-09-12T03:26:33.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.562+0000 7fc3f3e00640 1 -- 172.21.15.50:0/3281333703 wait complete. 2024-09-12T03:26:33.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 Processor -- start 2024-09-12T03:26:33.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 -- start start 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3f40745c0 0x7fc3f419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f4074ee0 0x7fc3f419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 0x7fc3f419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4113fb0 con 0x7fc3f40745c0 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4113e30 con 0x7fc3f4074ee0 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.563+0000 7fc3f3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4114130 con 0x7fc3f410e260 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3f40745c0 0x7fc3f419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 0x7fc3f419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 0x7fc3f419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45748/0 (socket says 172.21.15.50:45748) 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 -- 172.21.15.50:0/1530330067 learned_addr learned my addr 172.21.15.50:0/1530330067 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3ebe00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f4074ee0 0x7fc3f419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 -- 172.21.15.50:0/1530330067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f4074ee0 msgr2=0x7fc3f419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f4074ee0 0x7fc3f419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 -- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3f40745c0 msgr2=0x7fc3f419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3f40745c0 0x7fc3f419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.564+0000 7fc3f3400640 1 -- 172.21.15.50:0/1530330067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f419f450 con 0x7fc3f410e260 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.565+0000 7fc3f3400640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 0x7fc3f419ed70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c3f0 tx=0x7fc3ec00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.565+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec003970 con 0x7fc3f410e260 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.565+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ec00b690 con 0x7fc3f410e260 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.565+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec004420 con 0x7fc3f410e260 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.565+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f419f740 con 0x7fc3f410e260 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.565+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f41a7020 con 0x7fc3f410e260 2024-09-12T03:26:33.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.566+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f40749a0 con 0x7fc3f410e260 2024-09-12T03:26:33.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.568+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3ec01d020 con 0x7fc3f410e260 2024-09-12T03:26:33.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.569+0000 7fc3eb400640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc3d80a2c90 0x7fc3d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.569+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc3ec0cf370 con 0x7fc3f410e260 2024-09-12T03:26:33.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.569+0000 7fc3f2a00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc3d80a2c90 0x7fc3d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.570+0000 7fc3f2a00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc3d80a2c90 0x7fc3d80a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009bd0 tx=0x7fc3e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:33.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.573+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3ec09bd60 con 0x7fc3f410e260 2024-09-12T03:26:33.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.829+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f402d7c0 con 0x7fc3f410e260 2024-09-12T03:26:33.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.830+0000 7fc3eb400640 1 -- 172.21.15.50:0/1530330067 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fc3ec09b660 con 0x7fc3f410e260 2024-09-12T03:26:33.831 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:33.831 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"btime":"2024-09-12T03:25:24:859723+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:25:24.725697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24508,24526,24536]},"id":1}]} 2024-09-12T03:26:33.831 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:26:33.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc3d80a2c90 msgr2=0x7fc3d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc3d80a2c90 0x7fc3d80a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009bd0 tx=0x7fc3e4005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 msgr2=0x7fc3f419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 0x7fc3f419ed70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c3f0 tx=0x7fc3ec00c8b0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 shutdown_connections 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fc3d80a2c90 0x7fc3d80a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f410e260 0x7fc3f419ed70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f4074ee0 0x7fc3f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 --2- 172.21.15.50:0/1530330067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3f40745c0 0x7fc3f419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.833+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 >> 172.21.15.50:0/1530330067 conn(0x7fc3f40fd6e0 msgr2=0x7fc3f40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:33.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.834+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 shutdown_connections 2024-09-12T03:26:33.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:33.834+0000 7fc3f3e00640 1 -- 172.21.15.50:0/1530330067 wait complete. 2024-09-12T03:26:34.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:24:32.635781+0000', 'modified': '2024-09-12T03:25:24.725697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24526, 'mds_2': 24508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/1268137141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 1268137141}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 1268137141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/4279679346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 4279679346}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 4279679346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/809625574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 809625574}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 809625574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.142:6837/2093449893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2093449893}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2093449893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.142:6839/3589126725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3589126725}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3589126725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.82:6839/2348373979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2348373979}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2348373979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24508, 24526, 24536]} 2024-09-12T03:26:34.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:26:34.550 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T03:26:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:34 smithi050 ceph-mon[30378]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:34 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1530330067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:34.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:34 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:34.938 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:34 smithi142 ceph-mon[35329]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:34 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1530330067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:35.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:34 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:34 smithi082 ceph-mon[35046]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:34 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1530330067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:34 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 -- 172.21.15.50:0/1007537886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda8074510 msgr2=0x7fdda80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 --2- 172.21.15.50:0/1007537886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda8074510 0x7fdda80748f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009ca0 tx=0x7fdd9c02a950 comp rx=0 tx=0).stop 2024-09-12T03:26:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 -- 172.21.15.50:0/1007537886 shutdown_connections 2024-09-12T03:26:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 --2- 172.21.15.50:0/1007537886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda810e280 0x7fdda8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 --2- 172.21.15.50:0/1007537886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda8074e30 0x7fdda810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 --2- 172.21.15.50:0/1007537886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda8074510 0x7fdda80748f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 -- 172.21.15.50:0/1007537886 >> 172.21.15.50:0/1007537886 conn(0x7fdda80fd740 msgr2=0x7fdda80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 -- 172.21.15.50:0/1007537886 shutdown_connections 2024-09-12T03:26:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.561+0000 7fddaec00640 1 -- 172.21.15.50:0/1007537886 wait complete. 2024-09-12T03:26:35.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.562+0000 7fddaec00640 1 Processor -- start 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.562+0000 7fddaec00640 1 -- start start 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddaec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 0x7fdda819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddaec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda8074e30 0x7fdda819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddaec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda810e280 0x7fdda819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddaec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8113fb0 con 0x7fdda810e280 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddaec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8113e30 con 0x7fdda8074e30 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddaec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8114130 con 0x7fdda8074510 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 0x7fdda819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda810e280 0x7fdda819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 0x7fdda819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45770/0 (socket says 172.21.15.50:45770) 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 -- 172.21.15.50:0/3142889604 learned_addr learned my addr 172.21.15.50:0/3142889604 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:35.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddace00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda8074e30 0x7fdda819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 -- 172.21.15.50:0/3142889604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda8074e30 msgr2=0x7fdda819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda8074e30 0x7fdda819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 -- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda810e280 msgr2=0x7fdda819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda810e280 0x7fdda819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.563+0000 7fddad800640 1 -- 172.21.15.50:0/3142889604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda819f460 con 0x7fdda8074510 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.564+0000 7fddad800640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 0x7fdda819a4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c005cc0 tx=0x7fdd9c02f630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.564+0000 7fdda6000640 1 -- 172.21.15.50:0/3142889604 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c034040 con 0x7fdda8074510 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.564+0000 7fdda6000640 1 -- 172.21.15.50:0/3142889604 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd9c030920 con 0x7fdda8074510 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.564+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda819f6f0 con 0x7fdda8074510 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.564+0000 7fdda6000640 1 -- 172.21.15.50:0/3142889604 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c004780 con 0x7fdda8074510 2024-09-12T03:26:35.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.564+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda81a7030 con 0x7fdda8074510 2024-09-12T03:26:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.567+0000 7fddb2c9d740 1 Processor -- start 2024-09-12T03:26:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.567+0000 7fddb2c9d740 1 -- start start 2024-09-12T03:26:35.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.567+0000 7fdda6000640 1 -- 172.21.15.50:0/3142889604 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdd9c037020 con 0x7fdda8074510 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x5634472438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x563447250ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5634471a90b0 con 0x5634472434e0 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563447150ab0 con 0x563447243e00 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5634471a5070 con 0x56344724e6c0 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x5634472438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x5634472438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36552/0 (socket says 172.21.15.50:36552) 2024-09-12T03:26:35.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddae200640 1 -- 172.21.15.50:0/3582423780 learned_addr learned my addr 172.21.15.50:0/3582423780 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:35.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddad800640 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.568+0000 7fddace00640 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x563447250ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddae200640 1 -- 172.21.15.50:0/3582423780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 msgr2=0x563447250ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddae200640 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x563447250ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddae200640 1 -- 172.21.15.50:0/3582423780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 msgr2=0x56344724e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddae200640 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddae200640 1 -- 172.21.15.50:0/3582423780 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344715d4a0 con 0x5634472434e0 2024-09-12T03:26:35.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddad800640 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fddae200640 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x5634472438c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009ba0 tx=0x7fdda0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ec5c01096ae9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fdd9aa00640 1 -- 172.21.15.50:0/3582423780 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0004230 con 0x5634472434e0 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fdd9aa00640 1 -- 172.21.15.50:0/3582423780 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0009e10 con 0x5634472434e0 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fdd9aa00640 1 -- 172.21.15.50:0/3582423780 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0004d70 con 0x5634472434e0 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.569+0000 7fdda6000640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdd900a2c10 0x7fdd900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fdda6000640 1 -- 172.21.15.50:0/3142889604 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fdd9c0f0330 con 0x7fdda8074510 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddad800640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdd900a2c10 0x7fdd900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 -- 172.21.15.50:0/3582423780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 msgr2=0x5634472438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x5634472438c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009ba0 tx=0x7fdda0011810 comp rx=0 tx=0).stop 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 -- 172.21.15.50:0/3582423780 shutdown_connections 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x563447250ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/3582423780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x5634472438c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 -- 172.21.15.50:0/3582423780 >> 172.21.15.50:0/3582423780 conn(0x56344723fa70 msgr2=0x56344723fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 -- 172.21.15.50:0/3582423780 shutdown_connections 2024-09-12T03:26:35.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.570+0000 7fddb2c9d740 1 -- 172.21.15.50:0/3582423780 wait complete. 2024-09-12T03:26:35.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.571+0000 7fddad800640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdd900a2c10 0x7fdd900a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0f3fd0 tx=0x7fdd9c0f3de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.571+0000 7fddb2c9d740 1 -- start start 2024-09-12T03:26:35.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.571+0000 7fddb2c9d740 1 Processor -- start 2024-09-12T03:26:35.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x56344724b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563447243e00 0x56344724ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56344724e6c0 0x56344724bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563447251330 con 0x563447243e00 2024-09-12T03:26:35.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563447131920 con 0x56344724e6c0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5634472514b0 con 0x5634472434e0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddb2c9d740 10 client.0 _ll_register_callbacks cb 0x5634471558a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x56344724b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x56344724b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:45792/0 (socket says 172.21.15.50:45792) 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddace00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563447243e00 0x56344724ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 -- 172.21.15.50:0/556698342 learned_addr learned my addr 172.21.15.50:0/556698342 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56344724e6c0 0x56344724bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 -- 172.21.15.50:0/556698342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56344724e6c0 msgr2=0x56344724bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56344724e6c0 0x56344724bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 -- 172.21.15.50:0/556698342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563447243e00 msgr2=0x56344724ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563447243e00 0x56344724ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddae200640 1 -- 172.21.15.50:0/556698342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563446f1a2b0 con 0x5634472434e0 2024-09-12T03:26:35.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.572+0000 7fddace00640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563447243e00 0x56344724ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fddae200640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x56344724b4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009b70 tx=0x7fdda0004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb054d994641ce7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fdd8f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fdd8f400640 1 -- 172.21.15.50:0/556698342 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0003c80 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda0009e10 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fdd8f400640 1 -- 172.21.15.50:0/556698342 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fdda0003e20 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fdd8f400640 1 -- 172.21.15.50:0/556698342 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdda0029440 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fddb2c9d740 10 client.24982 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.573+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56344725cfa0 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fdd8f400640 1 -- 172.21.15.50:0/556698342 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0009e10 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fdd8f400640 1 -- 172.21.15.50:0/556698342 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdda0018070 con 0x5634472434e0 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fddb2c9d740 10 client.24982 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fddb2c9d740 10 client.24982 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fddb2c9d740 20 client.24982 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x56344725f3e0 0x563447260850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fddb2c9d740 4 client.24982 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:35.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.574+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 --> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634471b6620 con 0x56344725f3e0 2024-09-12T03:26:35.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.575+0000 7fddace00640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x56344725f3e0 0x563447260850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.575+0000 7fddace00640 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x56344725f3e0 0x563447260850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf3fd4d4125f665b server_cookie=98c5a346cbb0efc6 in_seq=0 out_seq=0 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.575+0000 7fdd8f400640 10 client.24982 ms_handle_connect on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.576+0000 7fdd8f400640 1 -- 172.21.15.50:0/556698342 <== mds.2 v2:172.21.15.142:6836/2093449893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634471b6620 con 0x56344725f3e0 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.576+0000 7fdd8f400640 10 client.24982 handle_command_reply: tid=0 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.576+0000 7fddb2c9d740 1 client.24982 shutdown 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 msgr2=0x56344724b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x56344724b4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009b70 tx=0x7fdda0004950 comp rx=0 tx=0).stop 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 shutdown_connections 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56344724e6c0 0x56344724bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x56344725f3e0 0x563447260850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563447243e00 0x56344724ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/556698342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x56344724b4e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fdd8f400640 0 client.24982 ms_handle_reset on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:35.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 >> 172.21.15.50:0/556698342 conn(0x56344723fa70 msgr2=0x56344723ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 shutdown_connections 2024-09-12T03:26:35.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 1 -- 172.21.15.50:0/556698342 wait complete. 2024-09-12T03:26:35.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.577+0000 7fddb2c9d740 20 client.24982 trim_cache size 0 max 16384 2024-09-12T03:26:35.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 Processor -- start 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 -- start start 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x563447260180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x56344724bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56344725eb90 con 0x5634472434e0 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56344725f260 con 0x563447243e00 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.596+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56344725f930 con 0x56344724e6c0 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x563447260180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x563447260180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36566/0 (socket says 172.21.15.50:36566) 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 -- 172.21.15.50:0/2779540037 learned_addr learned my addr 172.21.15.50:0/2779540037 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddace00640 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 -- 172.21.15.50:0/2779540037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 msgr2=0x56344724bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x56344724bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 -- 172.21.15.50:0/2779540037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 msgr2=0x56344724a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 -- 172.21.15.50:0/2779540037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344723f8a0 con 0x5634472434e0 2024-09-12T03:26:35.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddace00640 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fddae200640 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x563447260180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdda0000c00 tx=0x7fdda0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a155ff51a49d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.597+0000 7fdd9a000640 1 -- 172.21.15.50:0/2779540037 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00044f0 con 0x5634472434e0 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fdd9a000640 1 -- 172.21.15.50:0/2779540037 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0009780 con 0x5634472434e0 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fdd9a000640 1 -- 172.21.15.50:0/2779540037 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00295e0 con 0x5634472434e0 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 -- 172.21.15.50:0/2779540037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 msgr2=0x563447260180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x563447260180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdda0000c00 tx=0x7fdda0005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 -- 172.21.15.50:0/2779540037 shutdown_connections 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56344724e6c0 0x56344724bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x56344724a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/2779540037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5634472434e0 0x563447260180 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 -- 172.21.15.50:0/2779540037 >> 172.21.15.50:0/2779540037 conn(0x56344723fa70 msgr2=0x563447249db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 -- 172.21.15.50:0/2779540037 shutdown_connections 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.598+0000 7fddb2c9d740 1 -- 172.21.15.50:0/2779540037 wait complete. 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 -- start start 2024-09-12T03:26:35.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 Processor -- start 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x563447247eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x5634472483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56344724e6c0 0x5634472c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5634472606d0 con 0x56344724e6c0 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563447260000 con 0x563447243e00 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563447260da0 con 0x5634472434e0 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.599+0000 7fddb2c9d740 10 client.0 _ll_register_callbacks cb 0x7fdda0004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56344724e6c0 0x5634472c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddae200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56344724e6c0 0x5634472c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36576/0 (socket says 172.21.15.50:36576) 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddae200640 1 -- 172.21.15.50:0/1932136220 learned_addr learned my addr 172.21.15.50:0/1932136220 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x5634472483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddae200640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x563447247eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 -- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 msgr2=0x563447247eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x563447247eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 -- 172.21.15.50:0/1932136220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56344724e6c0 msgr2=0x5634472c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56344724e6c0 0x5634472c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 -- 172.21.15.50:0/1932136220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634472c8020 con 0x563447243e00 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fddace00640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x5634472483f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x56344724ad70 tx=0x7fdd94004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb96234aed795878 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.600+0000 7fdd8f400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fdd8f400640 1 -- 172.21.15.50:0/1932136220 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94004220 con 0x563447243e00 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda0009780 con 0x563447243e00 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fdd8f400640 1 -- 172.21.15.50:0/1932136220 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fdd940043c0 con 0x563447243e00 2024-09-12T03:26:35.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fdd8f400640 1 -- 172.21.15.50:0/1932136220 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd9402c050 con 0x563447243e00 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fddb2c9d740 10 client.24986 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634472c9200 con 0x563447243e00 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.601+0000 7fdd8f400640 1 -- 172.21.15.50:0/1932136220 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0009780 con 0x563447243e00 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fdd8f400640 1 -- 172.21.15.50:0/1932136220 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdd9401d660 con 0x563447243e00 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddb2c9d740 10 client.24986 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddb2c9d740 10 client.24986 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddb2c9d740 20 client.24986 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x5634472cdd70 0x5634472cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddb2c9d740 4 client.24986 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 --> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] -- command(tid 0: {"prefix": "status"}) -- 0x5634471b0b30 con 0x5634472cdd70 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.602+0000 7fddae200640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x5634472cdd70 0x5634472cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddae200640 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x5634472cdd70 0x5634472cca80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e751fd76526a6585 server_cookie=66ee6b7841eb2a83 in_seq=0 out_seq=0 2024-09-12T03:26:35.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fdd8f400640 10 client.24986 ms_handle_connect on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fdd8f400640 1 -- 172.21.15.50:0/1932136220 <== mds.2 v2:172.21.15.142:6836/2093449893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5634471b0b30 con 0x5634472cdd70 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fdd8f400640 10 client.24986 handle_command_reply: tid=0 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 client.24986 shutdown 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 msgr2=0x5634472483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x5634472483f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x56344724ad70 tx=0x7fdd94004d70 comp rx=0 tx=0).stop 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 shutdown_connections 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x5634472cdd70 0x5634472cca80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56344724e6c0 0x5634472c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563447243e00 0x5634472483f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 --2- 172.21.15.50:0/1932136220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634472434e0 0x563447247eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fdd8f400640 0 client.24986 ms_handle_reset on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 >> 172.21.15.50:0/1932136220 conn(0x56344723fa70 msgr2=0x56344724a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 shutdown_connections 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 1 -- 172.21.15.50:0/1932136220 wait complete. 2024-09-12T03:26:35.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.603+0000 7fddb2c9d740 20 client.24986 trim_cache size 0 max 16384 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24508, 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 73.750617000000005, 2024-09-12T03:26:35.608 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-09-12T03:26:35.609 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 94, 2024-09-12T03:26:35.609 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:26:35.609 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 150.598521538 2024-09-12T03:26:35.609 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:26:35.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdd900a2c10 msgr2=0x7fdd900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdd900a2c10 0x7fdd900a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0f3fd0 tx=0x7fdd9c0f3de0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 msgr2=0x7fdda819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 0x7fdda819a4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c005cc0 tx=0x7fdd9c02f630 comp rx=0 tx=0).stop 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 shutdown_connections 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fdd900a2c10 0x7fdd900a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda810e280 0x7fdda819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda8074e30 0x7fdda819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 --2- 172.21.15.50:0/3142889604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdda8074510 0x7fdda819a4b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 >> 172.21.15.50:0/3142889604 conn(0x7fdda80fd740 msgr2=0x7fdda80fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 shutdown_connections 2024-09-12T03:26:35.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:35.604+0000 7fddaec00640 1 -- 172.21.15.50:0/3142889604 wait complete. 2024-09-12T03:26:35.994 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:35.994 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:35.995 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:35 smithi050 ceph-mon[30378]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:36.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:36.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:35 smithi142 ceph-mon[35329]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:36.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:35 smithi082 ceph-mon[35046]: from='mgr.14152 172.21.15.50:0/3160746549' entity='mgr.x' 2024-09-12T03:26:36.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:26:36.719 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:36 smithi142 ceph-mon[35329]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:37.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:36 smithi050 ceph-mon[30378]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:36 smithi082 ceph-mon[35046]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 -- 172.21.15.50:0/629544093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87281079c0 msgr2=0x7f8728107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 --2- 172.21.15.50:0/629544093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87281079c0 0x7f8728107da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8718009930 tx=0x7f871802a420 comp rx=0 tx=0).stop 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 -- 172.21.15.50:0/629544093 shutdown_connections 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 --2- 172.21.15.50:0/629544093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8728102360 0x7f872810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 --2- 172.21.15.50:0/629544093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 0x7f8728101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 --2- 172.21.15.50:0/629544093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87281079c0 0x7f8728107da0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.303+0000 7f872da00640 1 -- 172.21.15.50:0/629544093 >> 172.21.15.50:0/629544093 conn(0x7f87280fd740 msgr2=0x7f87280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.304+0000 7f872da00640 1 -- 172.21.15.50:0/629544093 shutdown_connections 2024-09-12T03:26:37.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.304+0000 7f872da00640 1 -- 172.21.15.50:0/629544093 wait complete. 2024-09-12T03:26:37.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.305+0000 7f872da00640 1 Processor -- start 2024-09-12T03:26:37.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 -- start start 2024-09-12T03:26:37.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 0x7f872810cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8728102360 0x7f872810d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87281079c0 0x7f87281a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8728111dc0 con 0x7f8728102360 2024-09-12T03:26:37.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8728111c40 con 0x7f87281079c0 2024-09-12T03:26:37.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.306+0000 7f872da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8728111f40 con 0x7f87281019c0 2024-09-12T03:26:37.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8728102360 0x7f872810d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 0x7f872810cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8728102360 0x7f872810d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36588/0 (socket says 172.21.15.50:36588) 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f872d000640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87281079c0 0x7f87281a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727400640 1 -- 172.21.15.50:0/4161839865 learned_addr learned my addr 172.21.15.50:0/4161839865 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727e00640 1 -- 172.21.15.50:0/4161839865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87281079c0 msgr2=0x7f87281a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727e00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87281079c0 0x7f87281a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727e00640 1 -- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8728102360 msgr2=0x7f872810d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727e00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8728102360 0x7f872810d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.307+0000 7f8727e00640 1 -- 172.21.15.50:0/4161839865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87281a3940 con 0x7f87281019c0 2024-09-12T03:26:37.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.308+0000 7f8727e00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 0x7f872810cb10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8718005940 tx=0x7f87180040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.308+0000 7f8724c00640 1 -- 172.21.15.50:0/4161839865 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871803f070 con 0x7f87281019c0 2024-09-12T03:26:37.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.308+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87281a3bd0 con 0x7f87281019c0 2024-09-12T03:26:37.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.308+0000 7f8724c00640 1 -- 172.21.15.50:0/4161839865 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87180044c0 con 0x7f87281019c0 2024-09-12T03:26:37.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.309+0000 7f8724c00640 1 -- 172.21.15.50:0/4161839865 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718035070 con 0x7f87281019c0 2024-09-12T03:26:37.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.309+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87281ab4f0 con 0x7f87281019c0 2024-09-12T03:26:37.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.312+0000 7f87319e8740 1 Processor -- start 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.312+0000 7f87319e8740 1 -- start start 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.312+0000 7f8724c00640 1 -- 172.21.15.50:0/4161839865 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f871803b460 con 0x7f87281019c0 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f87319e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c03238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f87319e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f87319e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0330ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f87319e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5605c02890b0 con 0x5605c032e6c0 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f87319e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5605c0230ab0 con 0x5605c03234e0 2024-09-12T03:26:37.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f87319e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5605c0285070 con 0x5605c0323e00 2024-09-12T03:26:37.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f872d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c03238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f872d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c03238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:58016/0 (socket says 172.21.15.50:58016) 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f872d000640 1 -- 172.21.15.50:0/4190638822 learned_addr learned my addr 172.21.15.50:0/4190638822 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.313+0000 7f8727400640 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0330ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f872d000640 1 -- 172.21.15.50:0/4190638822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 msgr2=0x5605c032e180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f872d000640 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032e180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f872d000640 1 -- 172.21.15.50:0/4190638822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 msgr2=0x5605c0330ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f872d000640 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0330ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f872d000640 1 -- 172.21.15.50:0/4190638822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c023d4a0 con 0x5605c03234e0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f872d000640 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c03238c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8720009f90 tx=0x7f8720010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64971a0365febd4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f871d600640 1 -- 172.21.15.50:0/4190638822 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720011700 con 0x5605c03234e0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f871d600640 1 -- 172.21.15.50:0/4190638822 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8720011d80 con 0x5605c03234e0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f871d600640 1 -- 172.21.15.50:0/4190638822 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720018380 con 0x5605c03234e0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.314+0000 7f8724c00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f870c0a2c90 0x7f870c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f8724c00640 1 -- 172.21.15.50:0/4161839865 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f87180efa20 con 0x7f87281019c0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f8727e00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f870c0a2c90 0x7f870c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 -- 172.21.15.50:0/4190638822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 msgr2=0x5605c03238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c03238c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8720009f90 tx=0x7f8720010860 comp rx=0 tx=0).stop 2024-09-12T03:26:37.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 -- 172.21.15.50:0/4190638822 shutdown_connections 2024-09-12T03:26:37.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0330ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 --2- 172.21.15.50:0/4190638822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c03238c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 -- 172.21.15.50:0/4190638822 >> 172.21.15.50:0/4190638822 conn(0x5605c031fa70 msgr2=0x5605c031fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f8727e00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f870c0a2c90 0x7f870c0a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f87180c4450 tx=0x7f87180095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 -- 172.21.15.50:0/4190638822 shutdown_connections 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.315+0000 7f87319e8740 1 -- 172.21.15.50:0/4190638822 wait complete. 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.316+0000 7f87319e8740 1 -- start start 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.316+0000 7f87319e8740 1 Processor -- start 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 0x5605c032b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c032e6c0 0x5605c032c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5605c0331330 con 0x5605c03234e0 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5605c0211920 con 0x5605c032e6c0 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5605c03314b0 con 0x5605c0323e00 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f87319e8740 10 client.0 _ll_register_callbacks cb 0x5605c02358a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f872d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 0x5605c032b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f872d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 0x5605c032b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36606/0 (socket says 172.21.15.50:36606) 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f8727400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f872d000640 1 -- 172.21.15.50:0/596792597 learned_addr learned my addr 172.21.15.50:0/596792597 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.317+0000 7f872d000640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c032e6c0 0x5605c032c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8727400640 1 -- 172.21.15.50:0/596792597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c032e6c0 msgr2=0x5605c032c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8727400640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c032e6c0 0x5605c032c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8727400640 1 -- 172.21.15.50:0/596792597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 msgr2=0x5605c032b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8727400640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 0x5605c032b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8727400640 1 -- 172.21.15.50:0/596792597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605bfffa2b0 con 0x5605c0323e00 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f872d000640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 0x5605c032b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8727400640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032bad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87140126d0 tx=0x7f8714012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9937ec1667fc2ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8712000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8712000640 1 -- 172.21.15.50:0/596792597 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400f8c0 con 0x5605c0323e00 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f8712000640 1 -- 172.21.15.50:0/596792597 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f871400fa60 con 0x5605c0323e00 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.318+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8720011d80 con 0x5605c0323e00 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f8712000640 1 -- 172.21.15.50:0/596792597 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f871400fc60 con 0x5605c0323e00 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f87319e8740 10 client.24994 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:37.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605c033d080 con 0x5605c0323e00 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f8712000640 1 -- 172.21.15.50:0/596792597 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8720011d80 con 0x5605c0323e00 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f8712000640 1 -- 172.21.15.50:0/596792597 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f871401c420 con 0x5605c0323e00 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f87319e8740 10 client.24994 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.319+0000 7f87319e8740 10 client.24994 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f87319e8740 20 client.24994 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f87319e8740 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c033f720 0x5605c0340b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f87319e8740 4 client.24994 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 --> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605c0296620 con 0x5605c033f720 2024-09-12T03:26:37.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f872d000640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c033f720 0x5605c0340b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f872d000640 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c033f720 0x5605c0340b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f80ec07540ddbb3 server_cookie=c76ba8a33c6d8529 in_seq=0 out_seq=0 2024-09-12T03:26:37.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.320+0000 7f8712000640 10 client.24994 ms_handle_connect on v2:172.21.15.142:6838/3589126725 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.321+0000 7f8712000640 1 -- 172.21.15.50:0/596792597 <== mds.1 v2:172.21.15.142:6838/3589126725 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605c0296620 con 0x5605c033f720 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.321+0000 7f8712000640 10 client.24994 handle_command_reply: tid=0 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.321+0000 7f87319e8740 1 client.24994 shutdown 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 msgr2=0x5605c032bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032bad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87140126d0 tx=0x7f8714012b90 comp rx=0 tx=0).stop 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 shutdown_connections 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c033f720 0x5605c0340b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c032e6c0 0x5605c032c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c032bad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 --2- 172.21.15.50:0/596792597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c03234e0 0x5605c032b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 >> 172.21.15.50:0/596792597 conn(0x5605c031fa70 msgr2=0x5605c0325750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f8712000640 0 client.24994 ms_handle_reset on v2:172.21.15.142:6838/3589126725 2024-09-12T03:26:37.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 shutdown_connections 2024-09-12T03:26:37.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 1 -- 172.21.15.50:0/596792597 wait complete. 2024-09-12T03:26:37.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.322+0000 7f87319e8740 20 client.24994 trim_cache size 0 max 16384 2024-09-12T03:26:37.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 Processor -- start 2024-09-12T03:26:37.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 -- start start 2024-09-12T03:26:37.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0340b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c0325640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c032be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5605c033eed0 con 0x5605c032e6c0 2024-09-12T03:26:37.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5605c033f5a0 con 0x5605c03234e0 2024-09-12T03:26:37.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.341+0000 7f87319e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5605c033fc70 con 0x5605c0323e00 2024-09-12T03:26:37.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c032be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c032be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36614/0 (socket says 172.21.15.50:36614) 2024-09-12T03:26:37.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 -- 172.21.15.50:0/3831519503 learned_addr learned my addr 172.21.15.50:0/3831519503 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:37.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0340b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f8727400640 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c0325640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 -- 172.21.15.50:0/3831519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 msgr2=0x5605c0325640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c0325640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 -- 172.21.15.50:0/3831519503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 msgr2=0x5605c0340b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0340b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 -- 172.21.15.50:0/3831519503 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c0330e10 con 0x5605c032e6c0 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.342+0000 7f872d000640 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c032be60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8720009a80 tx=0x7f8720019a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d45c75c2b595db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.343+0000 7f871cc00640 1 -- 172.21.15.50:0/3831519503 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720019640 con 0x5605c032e6c0 2024-09-12T03:26:37.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.343+0000 7f871cc00640 1 -- 172.21.15.50:0/3831519503 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8720003e50 con 0x5605c032e6c0 2024-09-12T03:26:37.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.343+0000 7f871cc00640 1 -- 172.21.15.50:0/3831519503 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87200059c0 con 0x5605c032e6c0 2024-09-12T03:26:37.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 -- 172.21.15.50:0/3831519503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 msgr2=0x5605c032be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c032be60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8720009a80 tx=0x7f8720019a70 comp rx=0 tx=0).stop 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 -- 172.21.15.50:0/3831519503 shutdown_connections 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c032be60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c0325640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 --2- 172.21.15.50:0/3831519503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0340b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 -- 172.21.15.50:0/3831519503 >> 172.21.15.50:0/3831519503 conn(0x5605c031fa70 msgr2=0x5605c0324f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 -- 172.21.15.50:0/3831519503 shutdown_connections 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 -- 172.21.15.50:0/3831519503 wait complete. 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.344+0000 7f87319e8740 1 -- start start 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 Processor -- start 2024-09-12T03:26:37.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0342d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c03432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0343800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5605c0340a10 con 0x5605c032e6c0 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5605c0340340 con 0x5605c03234e0 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5605c03410e0 con 0x5605c0323e00 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f87319e8740 10 client.0 _ll_register_callbacks cb 0x7f8714009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f872d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0343800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f872d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0343800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36624/0 (socket says 172.21.15.50:36624) 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.345+0000 7f872d000640 1 -- 172.21.15.50:0/2634508783 learned_addr learned my addr 172.21.15.50:0/2634508783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c03432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f872d000640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0342d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 -- 172.21.15.50:0/2634508783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 msgr2=0x5605c0342d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0342d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 -- 172.21.15.50:0/2634508783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 msgr2=0x5605c0343800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0343800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 -- 172.21.15.50:0/2634508783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c03476a0 con 0x5605c0323e00 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8727400640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c03432c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f871401c890 tx=0x7f87140092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac62141204a1c137 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.346+0000 7f8712000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f8712000640 1 -- 172.21.15.50:0/2634508783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871402f060 con 0x5605c0323e00 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8720003e50 con 0x5605c0323e00 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f8712000640 1 -- 172.21.15.50:0/2634508783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f871401cb00 con 0x5605c0323e00 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f8712000640 1 -- 172.21.15.50:0/2634508783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f871401cd00 con 0x5605c0323e00 2024-09-12T03:26:37.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f87319e8740 10 client.25000 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605c03aa2d0 con 0x5605c0323e00 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f8712000640 1 -- 172.21.15.50:0/2634508783 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8720003e50 con 0x5605c0323e00 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f8712000640 1 -- 172.21.15.50:0/2634508783 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f87140048e0 con 0x5605c0323e00 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.347+0000 7f87319e8740 10 client.25000 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f87319e8740 10 client.25000 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f87319e8740 20 client.25000 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f87319e8740 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c03aedd0 0x5605c03adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f87319e8740 4 client.25000 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 --> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] -- command(tid 0: {"prefix": "status"}) -- 0x5605c0290b30 con 0x5605c03aedd0 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f872d000640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c03aedd0 0x5605c03adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f872d000640 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c03aedd0 0x5605c03adae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82e5f6ea03d56f74 server_cookie=cca26400c0208c09 in_seq=0 out_seq=0 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.348+0000 7f8712000640 10 client.25000 ms_handle_connect on v2:172.21.15.142:6838/3589126725 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f8712000640 1 -- 172.21.15.50:0/2634508783 <== mds.1 v2:172.21.15.142:6838/3589126725 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5605c0290b30 con 0x5605c03aedd0 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f8712000640 10 client.25000 handle_command_reply: tid=0 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 client.25000 shutdown 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 msgr2=0x5605c03432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c03432c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f871401c890 tx=0x7f87140092b0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 shutdown_connections 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x5605c03aedd0 0x5605c03adae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605c032e6c0 0x5605c0343800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605c0323e00 0x5605c03432c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 --2- 172.21.15.50:0/2634508783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5605c03234e0 0x5605c0342d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 >> 172.21.15.50:0/2634508783 conn(0x5605c031fa70 msgr2=0x5605c0329a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f8712000640 0 client.25000 ms_handle_reset on v2:172.21.15.142:6838/3589126725 2024-09-12T03:26:37.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 shutdown_connections 2024-09-12T03:26:37.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 1 -- 172.21.15.50:0/2634508783 wait complete. 2024-09-12T03:26:37.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.349+0000 7f87319e8740 20 client.25000 trim_cache size 0 max 16384 2024-09-12T03:26:37.355 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:26:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:26:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24526, 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 76.501258766999996, 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 94, 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 147.02173074199999 2024-09-12T03:26:37.356 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:26:37.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f870c0a2c90 msgr2=0x7f870c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f870c0a2c90 0x7f870c0a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f87180c4450 tx=0x7f87180095e0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 msgr2=0x7f872810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 0x7f872810cb10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8718005940 tx=0x7f87180040c0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 shutdown_connections 2024-09-12T03:26:37.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f870c0a2c90 0x7f870c0a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87281079c0 0x7f87281a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8728102360 0x7f872810d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 --2- 172.21.15.50:0/4161839865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87281019c0 0x7f872810cb10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 >> 172.21.15.50:0/4161839865 conn(0x7f87280fd740 msgr2=0x7f87280747c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 shutdown_connections 2024-09-12T03:26:37.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:37.350+0000 7f872da00640 1 -- 172.21.15.50:0/4161839865 wait complete. 2024-09-12T03:26:38.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:26:38.494 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:38.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:38 smithi050 ceph-mon[30378]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:38 smithi142 ceph-mon[35329]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 -- 172.21.15.50:0/3888385661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c074ee0 msgr2=0x7fa84c10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 --2- 172.21.15.50:0/3888385661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c074ee0 0x7fa84c10def0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa834009900 tx=0x7fa83402a480 comp rx=0 tx=0).stop 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 -- 172.21.15.50:0/3888385661 shutdown_connections 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 --2- 172.21.15.50:0/3888385661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa84c10e430 0x7fa84c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 --2- 172.21.15.50:0/3888385661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c074ee0 0x7fa84c10def0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 --2- 172.21.15.50:0/3888385661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c0745c0 0x7fa84c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 -- 172.21.15.50:0/3888385661 >> 172.21.15.50:0/3888385661 conn(0x7fa84c0fd8b0 msgr2=0x7fa84c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 -- 172.21.15.50:0/3888385661 shutdown_connections 2024-09-12T03:26:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.073+0000 7fa851000640 1 -- 172.21.15.50:0/3888385661 wait complete. 2024-09-12T03:26:39.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.074+0000 7fa851000640 1 Processor -- start 2024-09-12T03:26:39.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.074+0000 7fa851000640 1 -- start start 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa851000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 0x7fa84c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa851000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa84c074ee0 0x7fa84c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa851000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c10e430 0x7fa84c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa851000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c1141e0 con 0x7fa84c0745c0 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa851000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c114060 con 0x7fa84c074ee0 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa851000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c114360 con 0x7fa84c10e430 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa849e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 0x7fa84c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa849e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 0x7fa84c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40400/0 (socket says 172.21.15.50:40400) 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa849e00640 1 -- 172.21.15.50:0/4100007975 learned_addr learned my addr 172.21.15.50:0/4100007975 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa84a800640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c10e430 0x7fa84c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.075+0000 7fa843e00640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa84c074ee0 0x7fa84c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa849e00640 1 -- 172.21.15.50:0/4100007975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c10e430 msgr2=0x7fa84c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa849e00640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c10e430 0x7fa84c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa849e00640 1 -- 172.21.15.50:0/4100007975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa84c074ee0 msgr2=0x7fa84c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa849e00640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa84c074ee0 0x7fa84c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa849e00640 1 -- 172.21.15.50:0/4100007975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c19f630 con 0x7fa84c0745c0 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa84a800640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c10e430 0x7fa84c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa849e00640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 0x7fa84c19a680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00e9f0 tx=0x7fa83c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa842a00640 1 -- 172.21.15.50:0/4100007975 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00cd80 con 0x7fa84c0745c0 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa842a00640 1 -- 172.21.15.50:0/4100007975 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa83c005500 con 0x7fa84c0745c0 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c19f920 con 0x7fa84c0745c0 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa842a00640 1 -- 172.21.15.50:0/4100007975 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c010690 con 0x7fa84c0745c0 2024-09-12T03:26:39.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.076+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1a7200 con 0x7fa84c0745c0 2024-09-12T03:26:39.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.079+0000 7fa842a00640 1 -- 172.21.15.50:0/4100007975 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa83c0046c0 con 0x7fa84c0745c0 2024-09-12T03:26:39.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.079+0000 7fa853c1d740 1 Processor -- start 2024-09-12T03:26:39.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 -- start start 2024-09-12T03:26:39.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 0x55636463c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463cc50 0x556364646fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364649900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5563645a20b0 con 0x556364647510 2024-09-12T03:26:39.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556364549ab0 con 0x55636463c330 2024-09-12T03:26:39.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.080+0000 7fa853c1d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55636459e070 con 0x55636463cc50 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa84a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 0x55636463c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa84a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 0x55636463c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:60248/0 (socket says 172.21.15.50:60248) 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa849e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463cc50 0x556364646fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa843e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364649900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa84a800640 1 -- 172.21.15.50:0/895857957 learned_addr learned my addr 172.21.15.50:0/895857957 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa843e00640 1 -- 172.21.15.50:0/895857957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463cc50 msgr2=0x556364646fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa843e00640 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463cc50 0x556364646fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa843e00640 1 -- 172.21.15.50:0/895857957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 msgr2=0x55636463c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa843e00640 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 0x55636463c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa843e00640 1 -- 172.21.15.50:0/895857957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563645564a0 con 0x556364647510 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa84a800640 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 0x55636463c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa842a00640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa8300a2c90 0x7fa8300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.081+0000 7fa842a00640 1 -- 172.21.15.50:0/4100007975 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fa83c0cf9c0 con 0x7fa84c0745c0 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.082+0000 7fa843e00640 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364649900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa83402aa70 tx=0x7fa834005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b783a7036d4ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.082+0000 7fa84a800640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa8300a2c90 0x7fa8300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.082+0000 7fa83b400640 1 -- 172.21.15.50:0/895857957 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834035040 con 0x556364647510 2024-09-12T03:26:39.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.082+0000 7fa83b400640 1 -- 172.21.15.50:0/895857957 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa834004400 con 0x556364647510 2024-09-12T03:26:39.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.082+0000 7fa83b400640 1 -- 172.21.15.50:0/895857957 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340035f0 con 0x556364647510 2024-09-12T03:26:39.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 -- 172.21.15.50:0/895857957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 msgr2=0x556364649900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa84a800640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa8300a2c90 0x7fa8300a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa844009fd0 tx=0x7fa844009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364649900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa83402aa70 tx=0x7fa834005de0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 -- 172.21.15.50:0/895857957 shutdown_connections 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364649900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463cc50 0x556364646fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 --2- 172.21.15.50:0/895857957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463c330 0x55636463c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 -- 172.21.15.50:0/895857957 >> 172.21.15.50:0/895857957 conn(0x5563646388c0 msgr2=0x556364638ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 -- 172.21.15.50:0/895857957 shutdown_connections 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 -- 172.21.15.50:0/895857957 wait complete. 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.083+0000 7fa853c1d740 1 -- start start 2024-09-12T03:26:39.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 Processor -- start 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364644420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x556364644960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364644ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55636464a0c0 con 0x556364647510 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55636452a920 con 0x55636463cc50 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55636464a240 con 0x55636463c330 2024-09-12T03:26:39.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.084+0000 7fa853c1d740 10 client.0 _ll_register_callbacks cb 0x55636454e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa84a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364644ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364644420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa843e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x556364644960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364644420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32800/0 (socket says 172.21.15.50:32800) 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 -- 172.21.15.50:0/1516847783 learned_addr learned my addr 172.21.15.50:0/1516847783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 -- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 msgr2=0x556364644960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x556364644960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 -- 172.21.15.50:0/1516847783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 msgr2=0x556364644ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364644ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 -- 172.21.15.50:0/1516847783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563643132b0 con 0x55636463c330 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa849e00640 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364644420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa83c09b9b0 tx=0x7fa83c097f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c171c34721e69bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa82fe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.085+0000 7fa82fe00640 1 -- 172.21.15.50:0/1516847783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c09b610 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa834004400 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa82fe00640 1 -- 172.21.15.50:0/1516847783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fa83c09b300 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa82fe00640 1 -- 172.21.15.50:0/1516847783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa83c09c3b0 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 10 client.25012 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556364655e90 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa82fe00640 1 -- 172.21.15.50:0/1516847783 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa834004400 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa82fe00640 1 -- 172.21.15.50:0/1516847783 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa83c09ac00 con 0x55636463c330 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 10 client.25012 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 10 client.25012 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 20 client.25012 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x556364658450 0x5563646598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 4 client.25012 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.086+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563645af620 con 0x556364658450 2024-09-12T03:26:39.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.087+0000 7fa843e00640 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x556364658450 0x5563646598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.087+0000 7fa843e00640 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x556364658450 0x5563646598c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f264afc3970fa3c8 server_cookie=7ec4c244d44b4139 in_seq=0 out_seq=0 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.087+0000 7fa82fe00640 10 client.25012 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.088+0000 7fa82fe00640 1 -- 172.21.15.50:0/1516847783 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563645af620 con 0x556364658450 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.088+0000 7fa82fe00640 10 client.25012 handle_command_reply: tid=0 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.088+0000 7fa853c1d740 1 client.25012 shutdown 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 msgr2=0x556364644420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364644420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa83c09b9b0 tx=0x7fa83c097f50 comp rx=0 tx=0).stop 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 shutdown_connections 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x556364644ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x556364644960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x556364658450 0x5563646598c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 --2- 172.21.15.50:0/1516847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364644420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 >> 172.21.15.50:0/1516847783 conn(0x5563646388c0 msgr2=0x55636463e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa82fe00640 0 client.25012 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:26:39.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 shutdown_connections 2024-09-12T03:26:39.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 1 -- 172.21.15.50:0/1516847783 wait complete. 2024-09-12T03:26:39.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.089+0000 7fa853c1d740 20 client.25012 trim_cache size 0 max 16384 2024-09-12T03:26:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:38 smithi082 ceph-mon[35046]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 Processor -- start 2024-09-12T03:26:39.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 -- start start 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364657d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x55636463f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x55636463ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556364657c00 con 0x556364647510 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5563646582d0 con 0x55636463cc50 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa853c1d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5563646589a0 con 0x55636463c330 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa84a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x55636463ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa84a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x55636463ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40426/0 (socket says 172.21.15.50:40426) 2024-09-12T03:26:39.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa84a800640 1 -- 172.21.15.50:0/2182939109 learned_addr learned my addr 172.21.15.50:0/2182939109 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:39.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.108+0000 7fa843e00640 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x55636463f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa849e00640 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364657d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa84a800640 1 -- 172.21.15.50:0/2182939109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 msgr2=0x556364657d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa84a800640 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364657d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa84a800640 1 -- 172.21.15.50:0/2182939109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 msgr2=0x55636463f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa84a800640 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x55636463f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa84a800640 1 -- 172.21.15.50:0/2182939109 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556364643410 con 0x556364647510 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.109+0000 7fa84a800640 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x55636463ed90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa844011380 tx=0x7fa84401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9c35132853335 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa83aa00640 1 -- 172.21.15.50:0/2182939109 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84401e5b0 con 0x556364647510 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa83aa00640 1 -- 172.21.15.50:0/2182939109 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84401ec30 con 0x556364647510 2024-09-12T03:26:39.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa83aa00640 1 -- 172.21.15.50:0/2182939109 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844029db0 con 0x556364647510 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 -- 172.21.15.50:0/2182939109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 msgr2=0x55636463ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x55636463ed90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa844011380 tx=0x7fa84401a000 comp rx=0 tx=0).stop 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 -- 172.21.15.50:0/2182939109 shutdown_connections 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556364647510 0x55636463ed90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55636463cc50 0x55636463f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2182939109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364657d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 -- 172.21.15.50:0/2182939109 >> 172.21.15.50:0/2182939109 conn(0x5563646388c0 msgr2=0x556364645c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.110+0000 7fa853c1d740 1 -- 172.21.15.50:0/2182939109 shutdown_connections 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.111+0000 7fa853c1d740 1 -- 172.21.15.50:0/2182939109 wait complete. 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.111+0000 7fa853c1d740 1 -- start start 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.111+0000 7fa853c1d740 1 Processor -- start 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364661b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 0x5563646620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556364647510 0x55636465cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556364659740 con 0x55636463cc50 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556364659070 con 0x556364647510 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556364659e10 con 0x55636463c330 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa853c1d740 10 client.0 _ll_register_callbacks cb 0x7fa83c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa843e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 0x5563646620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa843e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 0x5563646620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40438/0 (socket says 172.21.15.50:40438) 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa843e00640 1 -- 172.21.15.50:0/2693914780 learned_addr learned my addr 172.21.15.50:0/2693914780 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa849e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364661b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa84a800640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556364647510 0x55636465cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa849e00640 1 -- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556364647510 msgr2=0x55636465cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa849e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556364647510 0x55636465cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa849e00640 1 -- 172.21.15.50:0/2693914780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 msgr2=0x5563646620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa849e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 0x5563646620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa849e00640 1 -- 172.21.15.50:0/2693914780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636465d080 con 0x55636463c330 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa843e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 0x5563646620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.112+0000 7fa84a800640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556364647510 0x55636465cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa849e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364661b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00cfd0 tx=0x7fa83c09b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f936e7b724053c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa82fe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa82fe00640 1 -- 172.21.15.50:0/2693914780 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0aa060 con 0x55636463c330 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84401ec30 con 0x55636463c330 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa82fe00640 1 -- 172.21.15.50:0/2693914780 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fa83c0a4d80 con 0x55636463c330 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa82fe00640 1 -- 172.21.15.50:0/2693914780 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa83c09b610 con 0x55636463c330 2024-09-12T03:26:39.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa853c1d740 10 client.25018 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563646c31b0 con 0x55636463c330 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.113+0000 7fa82fe00640 1 -- 172.21.15.50:0/2693914780 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84401ec30 con 0x55636463c330 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa82fe00640 1 -- 172.21.15.50:0/2693914780 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa83c0ba2e0 con 0x55636463c330 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa853c1d740 10 client.25018 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa853c1d740 10 client.25018 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa853c1d740 20 client.25018 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5563646c7bd0 0x5563646c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa853c1d740 4 client.25018 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "status"}) -- 0x5563645a9b30 con 0x5563646c7bd0 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.114+0000 7fa843e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5563646c7bd0 0x5563646c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa843e00640 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5563646c7bd0 0x5563646c68e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5630ce1fe3d1cc2 server_cookie=b836b2c93d4f94e5 in_seq=0 out_seq=0 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa82fe00640 10 client.25018 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa82fe00640 1 -- 172.21.15.50:0/2693914780 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5563645a9b30 con 0x5563646c7bd0 2024-09-12T03:26:39.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa82fe00640 10 client.25018 handle_command_reply: tid=0 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 client.25018 shutdown 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 msgr2=0x556364661b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364661b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00cfd0 tx=0x7fa83c09b330 comp rx=0 tx=0).stop 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 shutdown_connections 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556364647510 0x55636465cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55636463cc50 0x5563646620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5563646c7bd0 0x5563646c68e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 --2- 172.21.15.50:0/2693914780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55636463c330 0x556364661b90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 >> 172.21.15.50:0/2693914780 conn(0x5563646388c0 msgr2=0x556364637a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa82fe00640 0 client.25018 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 shutdown_connections 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 1 -- 172.21.15.50:0/2693914780 wait complete. 2024-09-12T03:26:39.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.115+0000 7fa853c1d740 20 client.25018 trim_cache size 0 max 16384 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24536, 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 125.478439689, 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 94, 2024-09-12T03:26:39.120 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:26:39.121 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 139.068932434 2024-09-12T03:26:39.121 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:26:39.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa8300a2c90 msgr2=0x7fa8300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa8300a2c90 0x7fa8300a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa844009fd0 tx=0x7fa844009500 comp rx=0 tx=0).stop 2024-09-12T03:26:39.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 msgr2=0x7fa84c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 0x7fa84c19a680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00e9f0 tx=0x7fa83c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 shutdown_connections 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa8300a2c90 0x7fa8300a5150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa84c10e430 0x7fa84c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa84c074ee0 0x7fa84c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 --2- 172.21.15.50:0/4100007975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa84c0745c0 0x7fa84c19a680 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 >> 172.21.15.50:0/4100007975 conn(0x7fa84c0fd8b0 msgr2=0x7fa84c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 shutdown_connections 2024-09-12T03:26:39.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:39.116+0000 7fa851000640 1 -- 172.21.15.50:0/4100007975 wait complete. 2024-09-12T03:26:39.869 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T03:26:39.869 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:26:39.879 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:26:39.879 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T03:26:40.261 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 -- 172.21.15.50:0/37334436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 msgr2=0x7fe8fc10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 --2- 172.21.15.50:0/37334436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc10a050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009ca0 tx=0x7fe8f402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 -- 172.21.15.50:0/37334436 shutdown_connections 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 --2- 172.21.15.50:0/37334436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc10a050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 --2- 172.21.15.50:0/37334436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 --2- 172.21.15.50:0/37334436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 -- 172.21.15.50:0/37334436 >> 172.21.15.50:0/37334436 conn(0x7fe8fc0fd390 msgr2=0x7fe8fc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 -- 172.21.15.50:0/37334436 shutdown_connections 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.800+0000 7fe904000640 1 -- 172.21.15.50:0/37334436 wait complete. 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.801+0000 7fe904000640 1 Processor -- start 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.801+0000 7fe904000640 1 -- start start 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.801+0000 7fe904000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.801+0000 7fe904000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe904000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe904000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10d860 con 0x7fe8fc1037c0 2024-09-12T03:26:40.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe904000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10d6e0 con 0x7fe8fc101c00 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe904000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10d9e0 con 0x7fe8fc102e00 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe901600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe902000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe902000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40464/0 (socket says 172.21.15.50:40464) 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe902000640 1 -- 172.21.15.50:0/1545931945 learned_addr learned my addr 172.21.15.50:0/1545931945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe901600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:60294/0 (socket says 172.21.15.50:60294) 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32836/0 (socket says 172.21.15.50:32836) 2024-09-12T03:26:40.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 -- 172.21.15.50:0/1545931945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 msgr2=0x7fe8fc075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 -- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 msgr2=0x7fe8fc074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.802+0000 7fe900c00640 1 -- 172.21.15.50:0/1545931945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc0753e0 con 0x7fe8fc102e00 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe901600640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe900c00640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc0745e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005fd0 tx=0x7fe8e800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8009c30 con 0x7fe8fc102e00 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc1a1fe0 con 0x7fe8fc102e00 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8009dd0 con 0x7fe8fc102e00 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8010460 con 0x7fe8fc102e00 2024-09-12T03:26:40.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.803+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a2440 con 0x7fe8fc102e00 2024-09-12T03:26:40.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.804+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc108f50 con 0x7fe8fc102e00 2024-09-12T03:26:40.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.808+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8e8003ef0 con 0x7fe8fc102e00 2024-09-12T03:26:40.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.809+0000 7fe8fa000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.809+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe8e80ceff0 con 0x7fe8fc102e00 2024-09-12T03:26:40.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.809+0000 7fe901600640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.809+0000 7fe901600640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0005fd0 tx=0x7fe8f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:40.813+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8e8017020 con 0x7fe8fc102e00 2024-09-12T03:26:41.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.012+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe8fc062630 con 0x7fe8e40a2c10 2024-09-12T03:26:41.013 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:40 smithi050 ceph-mon[30378]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:41.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.014+0000 7fe8fa000640 1 -- 172.21.15.50:0/1545931945 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe8fc062630 con 0x7fe8e40a2c10 2024-09-12T03:26:41.014 INFO:teuthology.orchestra.run.smithi050.stdout:Backend: cephadm 2024-09-12T03:26:41.014 INFO:teuthology.orchestra.run.smithi050.stdout:Available: Yes 2024-09-12T03:26:41.015 INFO:teuthology.orchestra.run.smithi050.stdout:Paused: No 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe8e40a2c10 msgr2=0x7fe8e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0005fd0 tx=0x7fe8f000b410 comp rx=0 tx=0).stop 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 msgr2=0x7fe8fc0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc0745e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005fd0 tx=0x7fe8e800dde0 comp rx=0 tx=0).stop 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 shutdown_connections 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8fc1037c0 0x7fe8fc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8fc102e00 0x7fe8fc0745e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:41.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 --2- 172.21.15.50:0/1545931945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc101c00 0x7fe8fc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 >> 172.21.15.50:0/1545931945 conn(0x7fe8fc0fd390 msgr2=0x7fe8fc106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 shutdown_connections 2024-09-12T03:26:41.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:41.016+0000 7fe904000640 1 -- 172.21.15.50:0/1545931945 wait complete. 2024-09-12T03:26:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:40 smithi142 ceph-mon[35329]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:40 smithi082 ceph-mon[35046]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:26:41.708 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T03:26:42.087 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:42.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 -- 172.21.15.50:0/2437581147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 msgr2=0x7f4184103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 --2- 172.21.15.50:0/2437581147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 0x7f4184103280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4170009900 tx=0x7f417002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 -- 172.21.15.50:0/2437581147 shutdown_connections 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 --2- 172.21.15.50:0/2437581147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41841037c0 0x7f418410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 --2- 172.21.15.50:0/2437581147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 0x7f4184103280 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 --2- 172.21.15.50:0/2437581147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f4184102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 -- 172.21.15.50:0/2437581147 >> 172.21.15.50:0/2437581147 conn(0x7f41840fd390 msgr2=0x7f41840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 -- 172.21.15.50:0/2437581147 shutdown_connections 2024-09-12T03:26:42.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.639+0000 7f418bc00640 1 -- 172.21.15.50:0/2437581147 wait complete. 2024-09-12T03:26:42.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.640+0000 7f418bc00640 1 Processor -- start 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.640+0000 7f418bc00640 1 -- start start 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f418bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f418419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f418bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 0x7f418419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f418bc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41841037c0 0x7f41841a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f418bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f418410d860 con 0x7f4184102e00 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f418bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f418410d6e0 con 0x7f4184101c00 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f418bc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f418410d9e0 con 0x7f41841037c0 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f418419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f418419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:60310/0 (socket says 172.21.15.50:60310) 2024-09-12T03:26:42.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 -- 172.21.15.50:0/2850793963 learned_addr learned my addr 172.21.15.50:0/2850793963 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4183e00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 0x7f418419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189c00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41841037c0 0x7f41841a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 -- 172.21.15.50:0/2850793963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41841037c0 msgr2=0x7f41841a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41841037c0 0x7f41841a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 -- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 msgr2=0x7f418419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 0x7f418419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.641+0000 7f4189200640 1 -- 172.21.15.50:0/2850793963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41841a1d90 con 0x7f4184101c00 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.642+0000 7f4189200640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f418419a160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f417800d950 tx=0x7f417800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.642+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178017020 con 0x7f4184101c00 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.642+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41841a2080 con 0x7f4184101c00 2024-09-12T03:26:42.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.642+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4178009c30 con 0x7f4184101c00 2024-09-12T03:26:42.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.642+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417800f6b0 con 0x7f4184101c00 2024-09-12T03:26:42.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.642+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841a2590 con 0x7f4184101c00 2024-09-12T03:26:42.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.643+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41841a2850 con 0x7f4184101c00 2024-09-12T03:26:42.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.645+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f417801a020 con 0x7f4184101c00 2024-09-12T03:26:42.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.647+0000 7f4181600640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f416c0a2c40 0x7f416c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.647+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f41780cf8d0 con 0x7f4184101c00 2024-09-12T03:26:42.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.647+0000 7f4183e00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f416c0a2c40 0x7f416c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.648+0000 7f4183e00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f416c0a2c40 0x7f416c0a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41700314e0 tx=0x7f4170000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.649+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4178021020 con 0x7f4184101c00 2024-09-12T03:26:42.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.873+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4184109cf0 con 0x7f416c0a2c40 2024-09-12T03:26:42.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:42 smithi050 ceph-mon[30378]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:42.874 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:42 smithi050 ceph-mon[30378]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:42.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.887+0000 7f4181600640 1 -- 172.21.15.50:0/2850793963 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f4184109cf0 con 0x7f416c0a2c40 2024-09-12T03:26:42.887 INFO:teuthology.orchestra.run.smithi050.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T03:26:42.887 INFO:teuthology.orchestra.run.smithi050.stdout:mds.a smithi050 running (2m) 8s ago 2m 13.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 2c1f08c25108 2024-09-12T03:26:42.887 INFO:teuthology.orchestra.run.smithi050.stdout:mds.b smithi082 running (2m) 10s ago 2m 26.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 5490d721def5 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.c smithi142 running (2m) 10s ago 2m 17.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 6b794d63593a 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.d smithi050 running (2m) 8s ago 2m 13.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 be85d7fc58f6 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.e smithi082 running (2m) 10s ago 2m 18.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ec81a90b9220 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.f smithi142 running (2m) 10s ago 2m 20.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 1c1bbd01c1d1 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.g smithi050 running (2m) 8s ago 2m 13.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 619790d41b10 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.h smithi082 running (2m) 10s ago 2m 18.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 fa8492535347 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.i smithi142 running (2m) 10s ago 2m 20.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 6a9a1ad592c8 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.j smithi050 running (2m) 8s ago 2m 13.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 c76b3d8286d2 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.k smithi082 running (2m) 10s ago 2m 14.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 53d8cbcad9db 2024-09-12T03:26:42.888 INFO:teuthology.orchestra.run.smithi050.stdout:mds.l smithi142 running (2m) 10s ago 2m 14.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad4f9db4c1cf 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.x smithi050 *:9283,8765 running (10m) 8s ago 10m 510M - 19.1.1-298-gb53b6d3e 54467abc55f8 d8126bff4f32 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.y smithi082 *:8443,8765 running (8m) 10s ago 8m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 349fd9d888ab 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.z smithi142 *:8443,8765 running (8m) 10s ago 8m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 cdaa4d580974 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:mon.a smithi050 running (10m) 8s ago 10m 50.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 606102499c9c 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:mon.b smithi082 running (8m) 10s ago 8m 41.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 21880aeea3ed 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:mon.c smithi142 running (8m) 10s ago 8m 41.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 53642584f51a 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 smithi050 running (8m) 8s ago 8m 45.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 59b19dc226f6 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 smithi082 running (7m) 10s ago 7m 46.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 f049cd9c18a0 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 smithi142 running (7m) 10s ago 7m 48.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cdbb3bd5d1ff 2024-09-12T03:26:42.889 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 smithi050 running (6m) 8s ago 6m 51.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 1e432eb2b0fa 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 smithi082 running (6m) 10s ago 6m 42.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cadf6e150936 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 smithi142 running (5m) 10s ago 5m 45.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 7da7e3ad2a3a 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 smithi050 running (5m) 8s ago 5m 48.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 0b94aa457bf6 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 smithi082 running (5m) 10s ago 5m 53.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 6da97a328ab3 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.8 smithi142 running (4m) 10s ago 4m 52.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 773052ed70f5 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.9 smithi050 running (4m) 8s ago 4m 41.4M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 339f5bb59d82 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.10 smithi082 running (3m) 10s ago 3m 55.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 a2c210e9e9a8 2024-09-12T03:26:42.890 INFO:teuthology.orchestra.run.smithi050.stdout:osd.11 smithi142 running (3m) 10s ago 3m 42.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cff3cd68b597 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f416c0a2c40 msgr2=0x7f416c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f416c0a2c40 0x7f416c0a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41700314e0 tx=0x7f4170000bc0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 msgr2=0x7f418419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f418419a160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f417800d950 tx=0x7f417800de10 comp rx=0 tx=0).stop 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 shutdown_connections 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f416c0a2c40 0x7f416c0a5100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41841037c0 0x7f41841a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4184102e00 0x7f418419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 --2- 172.21.15.50:0/2850793963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4184101c00 0x7f418419a160 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 >> 172.21.15.50:0/2850793963 conn(0x7f41840fd390 msgr2=0x7f41841085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 shutdown_connections 2024-09-12T03:26:42.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:42.890+0000 7f418bc00640 1 -- 172.21.15.50:0/2850793963 wait complete. 2024-09-12T03:26:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:42 smithi142 ceph-mon[35329]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:42 smithi142 ceph-mon[35329]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:42 smithi082 ceph-mon[35046]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:26:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:42 smithi082 ceph-mon[35046]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:43.602 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T03:26:43.983 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:44.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 -- 172.21.15.50:0/2999166056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08105810 msgr2=0x7f8b0810c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 --2- 172.21.15.50:0/2999166056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08105810 0x7f8b0810c0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009ca0 tx=0x7f8b0002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 -- 172.21.15.50:0/2999166056 shutdown_connections 2024-09-12T03:26:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 --2- 172.21.15.50:0/2999166056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08105810 0x7f8b0810c0a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 --2- 172.21.15.50:0/2999166056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b08104e50 0x7f8b081052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 --2- 172.21.15.50:0/2999166056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b08103c50 0x7f8b08104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 -- 172.21.15.50:0/2999166056 >> 172.21.15.50:0/2999166056 conn(0x7f8b080ff3e0 msgr2=0x7f8b08101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 -- 172.21.15.50:0/2999166056 shutdown_connections 2024-09-12T03:26:44.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.564+0000 7f8b0d400640 1 -- 172.21.15.50:0/2999166056 wait complete. 2024-09-12T03:26:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 Processor -- start 2024-09-12T03:26:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 -- start start 2024-09-12T03:26:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 0x7f8b08199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b08104e50 0x7f8b0819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b08105810 0x7f8b081a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810f920 con 0x7f8b08103c50 2024-09-12T03:26:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810f7a0 con 0x7f8b08105810 2024-09-12T03:26:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.565+0000 7f8b0d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0810faa0 con 0x7f8b08104e50 2024-09-12T03:26:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 0x7f8b08199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 0x7f8b08199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40506/0 (socket says 172.21.15.50:40506) 2024-09-12T03:26:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 -- 172.21.15.50:0/1034684595 learned_addr learned my addr 172.21.15.50:0/1034684595 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b0ca00640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b08105810 0x7f8b081a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b08104e50 0x7f8b0819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 -- 172.21.15.50:0/1034684595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b08104e50 msgr2=0x7f8b0819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b08104e50 0x7f8b0819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 -- 172.21.15.50:0/1034684595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b08105810 msgr2=0x7f8b081a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b08105810 0x7f8b081a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.566+0000 7f8b07e00640 1 -- 172.21.15.50:0/1034684595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b081a1b10 con 0x7f8b08103c50 2024-09-12T03:26:44.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.567+0000 7f8b07e00640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 0x7f8b08199ee0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8af800e9c0 tx=0x7f8af800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.567+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af800cde0 con 0x7f8b08103c50 2024-09-12T03:26:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.567+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af8010910 con 0x7f8b08103c50 2024-09-12T03:26:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.567+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8018a00 con 0x7f8b08103c50 2024-09-12T03:26:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.567+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b081a1e00 con 0x7f8b08103c50 2024-09-12T03:26:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.567+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b081a21a0 con 0x7f8b08103c50 2024-09-12T03:26:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.568+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0810b000 con 0x7f8b08103c50 2024-09-12T03:26:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.570+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8af801f020 con 0x7f8b08103c50 2024-09-12T03:26:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.571+0000 7f8b04c00640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8aec0a2bb0 0x7f8aec0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.572+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8af80cf5f0 con 0x7f8b08103c50 2024-09-12T03:26:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.572+0000 7f8b07400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8aec0a2bb0 0x7f8aec0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.572+0000 7f8b07400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8aec0a2bb0 0x7f8aec0a5070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b0819b400 tx=0x7f8af4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:44.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.575+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8af809c140 con 0x7f8b08103c50 2024-09-12T03:26:44.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.770+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8b08062630 con 0x7f8aec0a2bb0 2024-09-12T03:26:44.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.776+0000 7f8b04c00640 1 -- 172.21.15.50:0/1034684595 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8b08062630 con 0x7f8aec0a2bb0 2024-09-12T03:26:44.776 INFO:teuthology.orchestra.run.smithi050.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T03:26:44.776 INFO:teuthology.orchestra.run.smithi050.stdout:mds.all 12/12 12s ago 3m smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-12T03:26:44.776 INFO:teuthology.orchestra.run.smithi050.stdout:mgr 3/3 12s ago 8m smithi050=x;smithi082=y;smithi142=z;count:3 2024-09-12T03:26:44.776 INFO:teuthology.orchestra.run.smithi050.stdout:mon 3/3 12s ago 9m smithi050:172.21.15.50=a;smithi082:172.21.15.82=b;smithi142:172.21.15.142=c;count:3 2024-09-12T03:26:44.776 INFO:teuthology.orchestra.run.smithi050.stdout:osd 12 12s ago - 2024-09-12T03:26:44.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8aec0a2bb0 msgr2=0x7f8aec0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8aec0a2bb0 0x7f8aec0a5070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b0819b400 tx=0x7f8af4005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:44.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 msgr2=0x7f8b08199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 0x7f8b08199ee0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8af800e9c0 tx=0x7f8af800ee80 comp rx=0 tx=0).stop 2024-09-12T03:26:44.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 shutdown_connections 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f8aec0a2bb0 0x7f8aec0a5070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b08105810 0x7f8b081a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b08104e50 0x7f8b0819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 --2- 172.21.15.50:0/1034684595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b08103c50 0x7f8b08199ee0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 >> 172.21.15.50:0/1034684595 conn(0x7f8b080ff3e0 msgr2=0x7f8b081081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.778+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 shutdown_connections 2024-09-12T03:26:44.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:44.779+0000 7f8b0d400640 1 -- 172.21.15.50:0/1034684595 wait complete. 2024-09-12T03:26:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:44 smithi142 ceph-mon[35329]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:44 smithi142 ceph-mon[35329]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:45.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:44 smithi142 ceph-mon[35329]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T03:26:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:44 smithi082 ceph-mon[35046]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:44 smithi082 ceph-mon[35046]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:44 smithi082 ceph-mon[35046]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T03:26:45.127 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:44 smithi050 ceph-mon[30378]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:26:45.127 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:44 smithi050 ceph-mon[30378]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:45.127 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:44 smithi050 ceph-mon[30378]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T03:26:45.686 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T03:26:46.066 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:46.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 -- 172.21.15.50:0/2953871260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5470074b40 msgr2=0x7f5470074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:46.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 --2- 172.21.15.50:0/2953871260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5470074b40 0x7f5470074fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f545c009900 tx=0x7f545c02a410 comp rx=0 tx=0).stop 2024-09-12T03:26:46.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 -- 172.21.15.50:0/2953871260 shutdown_connections 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 --2- 172.21.15.50:0/2953871260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54701098b0 0x7f547010bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 --2- 172.21.15.50:0/2953871260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5470074b40 0x7f5470074fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 --2- 172.21.15.50:0/2953871260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54700761a0 0x7f5470074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 -- 172.21.15.50:0/2953871260 >> 172.21.15.50:0/2953871260 conn(0x7f54700fd1f0 msgr2=0x7f54700ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 -- 172.21.15.50:0/2953871260 shutdown_connections 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.623+0000 7f5476e00640 1 -- 172.21.15.50:0/2953871260 wait complete. 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 Processor -- start 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 -- start start 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5470074b40 0x7f5470199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54700761a0 0x7f547019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 0x7f54701a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f547010f8e0 con 0x7f54701098b0 2024-09-12T03:26:46.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f547010f760 con 0x7f5470074b40 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f547010fa60 con 0x7f54700761a0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 0x7f54701a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 0x7f54701a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40522/0 (socket says 172.21.15.50:40522) 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.624+0000 7f5476400640 1 -- 172.21.15.50:0/3867336584 learned_addr learned my addr 172.21.15.50:0/3867336584 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5475a00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5470074b40 0x7f5470199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5475000640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54700761a0 0x7f547019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5476400640 1 -- 172.21.15.50:0/3867336584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54700761a0 msgr2=0x7f547019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5476400640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54700761a0 0x7f547019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5476400640 1 -- 172.21.15.50:0/3867336584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5470074b40 msgr2=0x7f5470199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5476400640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5470074b40 0x7f5470199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5476400640 1 -- 172.21.15.50:0/3867336584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54701a1b30 con 0x7f54701098b0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.625+0000 7f5476400640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 0x7f54701a14c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5468009fd0 tx=0x7f546800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.626+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468017070 con 0x7f54701098b0 2024-09-12T03:26:46.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.626+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5468003e50 con 0x7f54701098b0 2024-09-12T03:26:46.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.626+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54680106c0 con 0x7f54701098b0 2024-09-12T03:26:46.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.626+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54701a1e20 con 0x7f54701098b0 2024-09-12T03:26:46.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.626+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54701a2220 con 0x7f54701098b0 2024-09-12T03:26:46.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.627+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54701086c0 con 0x7f54701098b0 2024-09-12T03:26:46.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.629+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5468003970 con 0x7f54701098b0 2024-09-12T03:26:46.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.631+0000 7f546e000640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54580a2c30 0x7f54580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:46.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.631+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f54680ceff0 con 0x7f54701098b0 2024-09-12T03:26:46.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.631+0000 7f5475a00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54580a2c30 0x7f54580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:46.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.632+0000 7f5475a00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54580a2c30 0x7f54580a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f546400bbe0 tx=0x7f546400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:46.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.635+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f546809b4f0 con 0x7f54701098b0 2024-09-12T03:26:46.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.827+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5470062630 con 0x7f54580a2c30 2024-09-12T03:26:46.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:46 smithi050 ceph-mon[30378]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:46.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:46 smithi050 ceph-mon[30378]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:26:46.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.828+0000 7f546e000640 1 -- 172.21.15.50:0/3867336584 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5470062630 con 0x7f54580a2c30 2024-09-12T03:26:46.828 INFO:teuthology.orchestra.run.smithi050.stdout:HOST ADDR LABELS STATUS 2024-09-12T03:26:46.828 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050 172.21.15.50 2024-09-12T03:26:46.828 INFO:teuthology.orchestra.run.smithi050.stdout:smithi082 172.21.15.82 2024-09-12T03:26:46.828 INFO:teuthology.orchestra.run.smithi050.stdout:smithi142 172.21.15.142 2024-09-12T03:26:46.829 INFO:teuthology.orchestra.run.smithi050.stdout:3 hosts in cluster 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54580a2c30 msgr2=0x7f54580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54580a2c30 0x7f54580a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f546400bbe0 tx=0x7f546400b390 comp rx=0 tx=0).stop 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 msgr2=0x7f54701a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 0x7f54701a14c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5468009fd0 tx=0x7f546800b260 comp rx=0 tx=0).stop 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 shutdown_connections 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f54580a2c30 0x7f54580a50f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54701098b0 0x7f54701a14c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54700761a0 0x7f547019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 --2- 172.21.15.50:0/3867336584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5470074b40 0x7f5470199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 >> 172.21.15.50:0/3867336584 conn(0x7f54700fd1f0 msgr2=0x7f5470105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 shutdown_connections 2024-09-12T03:26:46.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:46.831+0000 7f5476e00640 1 -- 172.21.15.50:0/3867336584 wait complete. 2024-09-12T03:26:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:46 smithi142 ceph-mon[35329]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:46 smithi142 ceph-mon[35329]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:26:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:46 smithi082 ceph-mon[35046]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:46 smithi082 ceph-mon[35046]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:26:47.509 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T03:26:47.889 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:48.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.416+0000 7fcbe5a00640 1 -- 172.21.15.50:0/3692306311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 msgr2=0x7fcbe00750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.416+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/3692306311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe00750b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009900 tx=0x7fcbcc02a410 comp rx=0 tx=0).stop 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 -- 172.21.15.50:0/3692306311 shutdown_connections 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/3692306311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbe010e1e0 0x7fcbe01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/3692306311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe00750b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/3692306311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe0076220 0x7fcbe0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 -- 172.21.15.50:0/3692306311 >> 172.21.15.50:0/3692306311 conn(0x7fcbe00fd390 msgr2=0x7fcbe00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 -- 172.21.15.50:0/3692306311 shutdown_connections 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.417+0000 7fcbe5a00640 1 -- 172.21.15.50:0/3692306311 wait complete. 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 Processor -- start 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 -- start start 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbe0076220 0x7fcbe019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe010e1e0 0x7fcbe01a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0113ea0 con 0x7fcbe0074c50 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0113d20 con 0x7fcbe010e1e0 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbe5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0114020 con 0x7fcbe0076220 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbdea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbdea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40542/0 (socket says 172.21.15.50:40542) 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbdea00640 1 -- 172.21.15.50:0/1596628902 learned_addr learned my addr 172.21.15.50:0/1596628902 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbdf400640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe010e1e0 0x7fcbe01a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.418+0000 7fcbde000640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbe0076220 0x7fcbe019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdea00640 1 -- 172.21.15.50:0/1596628902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbe0076220 msgr2=0x7fcbe019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdea00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbe0076220 0x7fcbe019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdea00640 1 -- 172.21.15.50:0/1596628902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe010e1e0 msgr2=0x7fcbe01a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdea00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe010e1e0 0x7fcbe01a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdea00640 1 -- 172.21.15.50:0/1596628902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe01a6120 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdf400640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe010e1e0 0x7fcbe01a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbdea00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe019e4f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000e9c0 tx=0x7fcbd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000cd80 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.419+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0005500 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.420+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe01a6410 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.420+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0010690 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.420+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe01a6920 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.421+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe01043b0 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.423+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbd00046c0 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.424+0000 7fcbd7400640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbc40a2bb0 0x7fcbc40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.424+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fcbd00cf7a0 con 0x7fcbe0074c50 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.424+0000 7fcbde000640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbc40a2bb0 0x7fcbc40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.425+0000 7fcbde000640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbc40a2bb0 0x7fcbc40a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0314e0 tx=0x7fcbcc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.427+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbd009c2f0 con 0x7fcbe0074c50 2024-09-12T03:26:48.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.620+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 --> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fcbe0062630 con 0x7fcbc40a2bb0 2024-09-12T03:26:48.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.622+0000 7fcbd7400640 1 -- 172.21.15.50:0/1596628902 <== mgr.14152 v2:172.21.15.50:6800/63385930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fcbe0062630 con 0x7fcbc40a2bb0 2024-09-12T03:26:48.622 INFO:teuthology.orchestra.run.smithi050.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T03:26:48.622 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-09-12T03:26:48.623 INFO:teuthology.orchestra.run.smithi050.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:26:48.623 INFO:teuthology.orchestra.run.smithi050.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:26:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.625+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbc40a2bb0 msgr2=0x7fcbc40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.625+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbc40a2bb0 0x7fcbc40a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0314e0 tx=0x7fcbcc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.625+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 msgr2=0x7fcbe019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.625+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe019e4f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000e9c0 tx=0x7fcbd000ee80 comp rx=0 tx=0).stop 2024-09-12T03:26:48.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 shutdown_connections 2024-09-12T03:26:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fcbc40a2bb0 0x7fcbc40a5070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbe010e1e0 0x7fcbe01a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbe0076220 0x7fcbe019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 --2- 172.21.15.50:0/1596628902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbe0074c50 0x7fcbe019e4f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 >> 172.21.15.50:0/1596628902 conn(0x7fcbe00fd390 msgr2=0x7fcbe0101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 shutdown_connections 2024-09-12T03:26:48.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:48.626+0000 7fcbe5a00640 1 -- 172.21.15.50:0/1596628902 wait complete. 2024-09-12T03:26:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:48 smithi050 ceph-mon[30378]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:26:48.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:48 smithi050 ceph-mon[30378]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:48 smithi142 ceph-mon[35329]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:26:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:48 smithi142 ceph-mon[35329]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:48 smithi082 ceph-mon[35046]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:26:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:48 smithi082 ceph-mon[35046]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:49.327 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:26:49.337 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:26:49.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T03:26:49.715 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:50.017 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:49 smithi050 ceph-mon[30378]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:50.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:49 smithi142 ceph-mon[35329]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:50.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:49 smithi082 ceph-mon[35046]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 -- 172.21.15.50:0/2305798698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34105810 msgr2=0x7f0c3410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 --2- 172.21.15.50:0/2305798698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34105810 0x7f0c3410c0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009ca0 tx=0x7f0c2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 -- 172.21.15.50:0/2305798698 shutdown_connections 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 --2- 172.21.15.50:0/2305798698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34105810 0x7f0c3410c0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 --2- 172.21.15.50:0/2305798698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c34104e50 0x7f0c341052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 --2- 172.21.15.50:0/2305798698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c34103c50 0x7f0c34104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 -- 172.21.15.50:0/2305798698 >> 172.21.15.50:0/2305798698 conn(0x7f0c340ff3e0 msgr2=0x7f0c34101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 -- 172.21.15.50:0/2305798698 shutdown_connections 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.249+0000 7f0c33200640 1 -- 172.21.15.50:0/2305798698 wait complete. 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 Processor -- start 2024-09-12T03:26:50.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 -- start start 2024-09-12T03:26:50.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 0x7f0c34199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c34104e50 0x7f0c3419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c34105810 0x7f0c341a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3410f8b0 con 0x7f0c34105810 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3410f730 con 0x7f0c34103c50 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.250+0000 7f0c33200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3410fa30 con 0x7f0c34104e50 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 0x7f0c34199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 0x7f0c34199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57574/0 (socket says 172.21.15.50:57574) 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 -- 172.21.15.50:0/795703081 learned_addr learned my addr 172.21.15.50:0/795703081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c32800640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c34105810 0x7f0c341a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c2be00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c34104e50 0x7f0c3419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 -- 172.21.15.50:0/795703081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c34104e50 msgr2=0x7f0c3419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c34104e50 0x7f0c3419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 -- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c34105810 msgr2=0x7f0c341a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c34105810 0x7f0c341a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 -- 172.21.15.50:0/795703081 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c341a1ae0 con 0x7f0c34103c50 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.251+0000 7f0c31e00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 0x7f0c34199eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9c0 tx=0x7f0c2400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.252+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400cd80 con 0x7f0c34103c50 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.252+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c240035f0 con 0x7f0c34103c50 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.252+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24010690 con 0x7f0c34103c50 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.252+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c341a1dd0 con 0x7f0c34103c50 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.252+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341a2170 con 0x7f0c34103c50 2024-09-12T03:26:50.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.253+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3410b050 con 0x7f0c34103c50 2024-09-12T03:26:50.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.255+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c240037a0 con 0x7f0c34103c50 2024-09-12T03:26:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.256+0000 7f0c2aa00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0c180a2bb0 0x7f0c180a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.256+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f0c240cf860 con 0x7f0c34103c50 2024-09-12T03:26:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.257+0000 7f0c2be00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0c180a2bb0 0x7f0c180a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.257+0000 7f0c2be00640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0c180a2bb0 0x7f0c180a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c3419b3d0 tx=0x7f0c1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.258+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c2409c440 con 0x7f0c34103c50 2024-09-12T03:26:50.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.479+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0c3402da50 con 0x7f0c34103c50 2024-09-12T03:26:50.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.480+0000 7f0c2aa00640 1 -- 172.21.15.50:0/795703081 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3088 (secure 0 0 0) 0x7f0c2409bd40 con 0x7f0c34103c50 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:e25 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:btime 2024-09-12T03:25:24:859723+0000 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:legacy client fscid: 1 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:Filesystem 'cephfs' (1) 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:fs_name cephfs 2024-09-12T03:26:50.481 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 25 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-09-12T03:24:32.635781+0000 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2024-09-12T03:25:24.725697+0000 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:tableserver 0 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:root 0 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:session_timeout 300 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:session_autoclose 300 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:max_file_size 1099511627776 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:max_xattr_size 65536 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:required_client_features {} 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:last_failure 0 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:last_failure_osd_epoch 0 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:max_mds 3 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:in 0,1,2 2024-09-12T03:26:50.482 INFO:teuthology.orchestra.run.smithi050.stdout:up {0=24536,1=24526,2=24508} 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:failed 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:damaged 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:stopped 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:data_pools [3,4] 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:metadata_pool 2 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:inline_data disabled 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:balancer 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:bal_rank_mask -1 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:standby_count_wanted 0 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:qdb_cluster leader: 24536 members: 24508,24526,24536 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b{0:24536} state up:active seq 5 addr [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.f{0:24502} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/809625574,v1:172.21.15.142:6835/809625574] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.i{1:24526} state up:active seq 19 addr [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.h{1:24500} state up:standby-replay seq 1 addr [v2:172.21.15.82:6836/4279679346,v1:172.21.15.82:6837/4279679346] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.483 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c{2:24508} state up:active seq 21 addr [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.e{2:24479} state up:standby-replay seq 1 addr [v2:172.21.15.82:6834/1268137141,v1:172.21.15.82:6835/1268137141] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:Standby daemons: 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.j{-1:14448} state up:standby seq 1 addr [v2:172.21.15.50:6834/3398998523,v1:172.21.15.50:6835/3398998523] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.g{-1:14496} state up:standby seq 1 addr [v2:172.21.15.50:6838/3736856191,v1:172.21.15.50:6839/3736856191] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a{-1:14508} state up:standby seq 1 addr [v2:172.21.15.50:6840/2806488600,v1:172.21.15.50:6841/2806488600] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.82:6832/196275210,v1:172.21.15.82:6833/196275210] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.142:6832/1399602993,v1:172.21.15.142:6833/1399602993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.484 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d{-1:24467} state up:standby seq 1 addr [v2:172.21.15.50:6836/3806772200,v1:172.21.15.50:6837/3806772200] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:26:50.485 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:26:50.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0c180a2bb0 msgr2=0x7f0c180a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0c180a2bb0 0x7f0c180a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c3419b3d0 tx=0x7f0c1c005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:50.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 msgr2=0x7f0c34199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 0x7f0c34199eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9c0 tx=0x7f0c2400ee80 comp rx=0 tx=0).stop 2024-09-12T03:26:50.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 shutdown_connections 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f0c180a2bb0 0x7f0c180a5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c34105810 0x7f0c341a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c34104e50 0x7f0c3419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 --2- 172.21.15.50:0/795703081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c34103c50 0x7f0c34199eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 >> 172.21.15.50:0/795703081 conn(0x7f0c340ff3e0 msgr2=0x7f0c34108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 shutdown_connections 2024-09-12T03:26:50.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:50.483+0000 7f0c33200640 1 -- 172.21.15.50:0/795703081 wait complete. 2024-09-12T03:26:51.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:50 smithi142 ceph-mon[35329]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:51.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:50 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/795703081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:26:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:50 smithi082 ceph-mon[35046]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:50 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/795703081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:26:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:50 smithi050 ceph-mon[30378]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:50 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/795703081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:26:51.281 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T03:26:51.656 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:52.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:51 smithi142 ceph-mon[35329]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:26:52.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:51 smithi082 ceph-mon[35046]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:26:52.108 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:51 smithi050 ceph-mon[30378]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.223+0000 7f2918200640 1 -- 172.21.15.50:0/987777681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 msgr2=0x7f2910104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.223+0000 7f2918200640 1 --2- 172.21.15.50:0/987777681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f2910104200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2904009ca0 tx=0x7f290402a8b0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 -- 172.21.15.50:0/987777681 shutdown_connections 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 --2- 172.21.15.50:0/987777681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910105920 0x7f291010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 --2- 172.21.15.50:0/987777681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910104f60 0x7f29101053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 --2- 172.21.15.50:0/987777681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f2910104200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 -- 172.21.15.50:0/987777681 >> 172.21.15.50:0/987777681 conn(0x7f29100ff5b0 msgr2=0x7f29101019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:52.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 -- 172.21.15.50:0/987777681 shutdown_connections 2024-09-12T03:26:52.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.224+0000 7f2918200640 1 -- 172.21.15.50:0/987777681 wait complete. 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 Processor -- start 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 -- start start 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f291019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910104f60 0x7f291019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910105920 0x7f29101a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f291010fb00 con 0x7f2910103e00 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f291010f980 con 0x7f2910105920 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2918200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f291010fc80 con 0x7f2910104f60 2024-09-12T03:26:52.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2915800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f291019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.225+0000 7f2915800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f291019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53138/0 (socket says 172.21.15.50:53138) 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2914e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910104f60 0x7f291019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 -- 172.21.15.50:0/3478559511 learned_addr learned my addr 172.21.15.50:0/3478559511 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2916200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910105920 0x7f29101a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 -- 172.21.15.50:0/3478559511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910104f60 msgr2=0x7f291019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910104f60 0x7f291019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 -- 172.21.15.50:0/3478559511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910105920 msgr2=0x7f29101a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910105920 0x7f29101a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 -- 172.21.15.50:0/3478559511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29101a1d00 con 0x7f2910103e00 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2914e00640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910104f60 0x7f291019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2916200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910105920 0x7f29101a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:52.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.226+0000 7f2915800640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f291019a0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2904000f80 tx=0x7f2904003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.227+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904004040 con 0x7f2910103e00 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.227+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904040070 con 0x7f2910103e00 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.227+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904045d40 con 0x7f2910103e00 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.227+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29101a1f90 con 0x7f2910103e00 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.227+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101a2440 con 0x7f2910103e00 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.228+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291010b1a0 con 0x7f2910103e00 2024-09-12T03:26:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.230+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2904045470 con 0x7f2910103e00 2024-09-12T03:26:52.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.231+0000 7f290e000640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f28f80a2c30 0x7f28f80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:52.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.231+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f29040f0590 con 0x7f2910103e00 2024-09-12T03:26:52.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.231+0000 7f2914e00640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f28f80a2c30 0x7f28f80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:52.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.232+0000 7f2914e00640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f28f80a2c30 0x7f28f80a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f291019b5f0 tx=0x7f28fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.235+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29040bd200 con 0x7f2910103e00 2024-09-12T03:26:52.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.424+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2910062e20 con 0x7f2910103e00 2024-09-12T03:26:52.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.425+0000 7f290e000640 1 -- 172.21.15.50:0/3478559511 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5656 (secure 0 0 0) 0x7f29040bcb00 con 0x7f2910103e00 2024-09-12T03:26:52.425 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 94 2024-09-12T03:26:52.425 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-09-12T03:16:17.849714+0000 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2024-09-12T03:26:14.656446+0000 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:crush_version 27 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:full_ratio 0.95 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:backfillfull_ratio 0.9 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:nearfull_ratio 0.85 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:require_min_compat_client luminous 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:min_compat_client luminous 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:require_osd_release squid 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:stretch_mode_enabled false 2024-09-12T03:26:52.426 INFO:teuthology.orchestra.run.smithi050.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:max_osd 12 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6802/894870617,v1:172.21.15.50:6803/894870617] [v2:172.21.15.50:6804/894870617,v1:172.21.15.50:6805/894870617] exists,up 650c6e13-7a10-4d87-abd0-ed5b66b22521 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6800/987070931,v1:172.21.15.82:6801/987070931] [v2:172.21.15.82:6802/987070931,v1:172.21.15.82:6803/987070931] exists,up 989d2e6e-def7-4a97-a83f-da0168e02a90 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/1535997813,v1:172.21.15.142:6801/1535997813] [v2:172.21.15.142:6802/1535997813,v1:172.21.15.142:6803/1535997813] exists,up b5565f10-86c0-4440-9256-ae2f1729cfb6 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6810/2709499694,v1:172.21.15.50:6811/2709499694] [v2:172.21.15.50:6812/2709499694,v1:172.21.15.50:6813/2709499694] exists,up a331fa56-9a0a-4823-bd51-3d23adb41a13 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 up in weight 1 up_from 32 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6808/2469208064,v1:172.21.15.82:6809/2469208064] [v2:172.21.15.82:6810/2469208064,v1:172.21.15.82:6811/2469208064] exists,up 61ff9884-cde7-4179-836b-0acc8b7a0e2b 2024-09-12T03:26:52.427 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/361252032,v1:172.21.15.142:6809/361252032] [v2:172.21.15.142:6810/361252032,v1:172.21.15.142:6811/361252032] exists,up 0636aa4c-98aa-4b31-86ea-d77fc89be048 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 up in weight 1 up_from 44 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6818/3669387354,v1:172.21.15.50:6819/3669387354] [v2:172.21.15.50:6820/3669387354,v1:172.21.15.50:6821/3669387354] exists,up cd0f4cc5-c370-4f89-8332-b28096839552 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 up in weight 1 up_from 49 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6816/704851047,v1:172.21.15.82:6817/704851047] [v2:172.21.15.82:6818/704851047,v1:172.21.15.82:6819/704851047] exists,up a73508ea-e10f-466a-8ce6-fcdffbb2f047 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:osd.8 up in weight 1 up_from 56 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/1190406046,v1:172.21.15.142:6817/1190406046] [v2:172.21.15.142:6818/1190406046,v1:172.21.15.142:6819/1190406046] exists,up 1fe07bf7-a040-4ddc-b175-de41dae683c5 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:osd.9 up in weight 1 up_from 64 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6826/1350383902,v1:172.21.15.50:6827/1350383902] [v2:172.21.15.50:6828/1350383902,v1:172.21.15.50:6829/1350383902] exists,up be1e8d37-b1a4-4f1f-a019-efa72dcaacaa 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:osd.10 up in weight 1 up_from 70 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6824/1817816537,v1:172.21.15.82:6825/1817816537] [v2:172.21.15.82:6826/1817816537,v1:172.21.15.82:6827/1817816537] exists,up 2660198c-5848-4738-910b-711c291758a5 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:osd.11 up in weight 1 up_from 77 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/572198399,v1:172.21.15.142:6825/572198399] [v2:172.21.15.142:6826/572198399,v1:172.21.15.142:6827/572198399] exists,up 7d5befe3-b567-4cd2-b31a-a8d06d402568 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.c [2,4] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.f [8,11] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.10 [2,1] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.1c [8,11] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.3 [4,9] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.17 [4,6] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.27 [3,9] 2024-09-12T03:26:52.428 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.36 [4,9] 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.37 [5,8] 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.2 [9,1] 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.b [9,8] 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.f [9,8] 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.1c [5,10] 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3330091767 expires 2024-09-13T03:17:12.653017+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6800/2850856182 expires 2024-09-13T03:17:12.653017+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/1363145359 expires 2024-09-13T03:17:12.653017+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3574998620 expires 2024-09-13T03:17:12.653017+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/2127938282 expires 2024-09-13T03:16:45.524611+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6801/1939431272 expires 2024-09-13T03:16:45.524611+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6801/2850856182 expires 2024-09-13T03:17:12.653017+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6800/1939431272 expires 2024-09-13T03:16:45.524611+0000 2024-09-12T03:26:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/1226501005 expires 2024-09-13T03:16:45.524611+0000 2024-09-12T03:26:52.430 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/1765792264 expires 2024-09-13T03:16:45.524611+0000 2024-09-12T03:26:52.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f28f80a2c30 msgr2=0x7f28f80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f28f80a2c30 0x7f28f80a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f291019b5f0 tx=0x7f28fc005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:52.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 msgr2=0x7f291019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f291019a0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2904000f80 tx=0x7f2904003ba0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 shutdown_connections 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f28f80a2c30 0x7f28f80a50f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910105920 0x7f29101a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2910104f60 0x7f291019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 --2- 172.21.15.50:0/3478559511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2910103e00 0x7f291019a0d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 >> 172.21.15.50:0/3478559511 conn(0x7f29100ff5b0 msgr2=0x7f2910108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 shutdown_connections 2024-09-12T03:26:52.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:52.428+0000 7f2918200640 1 -- 172.21.15.50:0/3478559511 wait complete. 2024-09-12T03:26:53.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:52 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3478559511' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:26:53.088 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:52 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3478559511' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:26:53.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:52 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3478559511' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:26:53.128 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T03:26:53.345 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:53.723 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:54.031 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:53 smithi050 ceph-mon[30378]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:54.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:53 smithi142 ceph-mon[35329]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:54.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:53 smithi082 ceph-mon[35046]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:54.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.280+0000 7f5898a00640 1 -- 172.21.15.50:0/565738027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102790 msgr2=0x7f58941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.280+0000 7f5898a00640 1 --2- 172.21.15.50:0/565738027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102790 0x7f58941067e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f587c009900 tx=0x7f587c02a410 comp rx=0 tx=0).stop 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 -- 172.21.15.50:0/565738027 shutdown_connections 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 --2- 172.21.15.50:0/565738027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5894106f10 0x7f589410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 --2- 172.21.15.50:0/565738027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102790 0x7f58941067e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 --2- 172.21.15.50:0/565738027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894101de0 0x7f58941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 -- 172.21.15.50:0/565738027 >> 172.21.15.50:0/565738027 conn(0x7f58940fd8c0 msgr2=0x7f58940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 -- 172.21.15.50:0/565738027 shutdown_connections 2024-09-12T03:26:54.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.281+0000 7f5898a00640 1 -- 172.21.15.50:0/565738027 wait complete. 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 Processor -- start 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 -- start start 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894101de0 0x7f58940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 0x7f5894074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5894106f10 0x7f5894074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5894111fc0 con 0x7f5894101de0 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5894111e40 con 0x7f5894102790 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.282+0000 7f5898a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5894112140 con 0x7f5894106f10 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f5891e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894101de0 0x7f58940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 0x7f5894074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 0x7f5894074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57602/0 (socket says 172.21.15.50:57602) 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 -- 172.21.15.50:0/3560757164 learned_addr learned my addr 172.21.15.50:0/3560757164 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:54.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f5892800640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5894106f10 0x7f5894074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:54.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 -- 172.21.15.50:0/3560757164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5894106f10 msgr2=0x7f5894074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:54.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5894106f10 0x7f5894074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 -- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894101de0 msgr2=0x7f58940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:54.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894101de0 0x7f58940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 -- 172.21.15.50:0/3560757164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58941a68c0 con 0x7f5894102790 2024-09-12T03:26:54.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f5891e00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894101de0 0x7f58940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.283+0000 7f588be00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 0x7f5894074a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f587c004770 tx=0x7f587c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.284+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c03f070 con 0x7f5894102790 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.284+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58941a6b50 con 0x7f5894102790 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.284+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c031ca0 con 0x7f5894102790 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.284+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c035030 con 0x7f5894102790 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.284+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58941a7080 con 0x7f5894102790 2024-09-12T03:26:54.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.285+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58941021c0 con 0x7f5894102790 2024-09-12T03:26:54.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.287+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f587c042070 con 0x7f5894102790 2024-09-12T03:26:54.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.288+0000 7f588aa00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58780a2c00 0x7f58780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:54.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.288+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f587c0eff30 con 0x7f5894102790 2024-09-12T03:26:54.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.289+0000 7f5891e00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58780a2c00 0x7f58780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:54.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.289+0000 7f5891e00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58780a2c00 0x7f58780a50c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58840095f0 tx=0x7f5884005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:54.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.291+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f587c0bc4f0 con 0x7f5894102790 2024-09-12T03:26:54.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.511+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5894075a20 con 0x7f5894102790 2024-09-12T03:26:54.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.512+0000 7f588aa00640 1 -- 172.21.15.50:0/3560757164 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f5894075a20 con 0x7f5894102790 2024-09-12T03:26:54.512 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:54.512 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"btime":"2024-09-12T03:25:24:859723+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:25:24.725697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24508,24526,24536]},"id":1}]} 2024-09-12T03:26:54.512 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:26:54.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58780a2c00 msgr2=0x7f58780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:54.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58780a2c00 0x7f58780a50c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58840095f0 tx=0x7f5884005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:54.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 msgr2=0x7f5894074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:54.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 0x7f5894074a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f587c004770 tx=0x7f587c003c10 comp rx=0 tx=0).stop 2024-09-12T03:26:54.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 shutdown_connections 2024-09-12T03:26:54.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f58780a2c00 0x7f58780a50c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5894106f10 0x7f5894074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5894102790 0x7f5894074a30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 --2- 172.21.15.50:0/3560757164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894101de0 0x7f58940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:54.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.514+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 >> 172.21.15.50:0/3560757164 conn(0x7f58940fd8c0 msgr2=0x7f58940fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:54.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.515+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 shutdown_connections 2024-09-12T03:26:54.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:54.515+0000 7f5898a00640 1 -- 172.21.15.50:0/3560757164 wait complete. 2024-09-12T03:26:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:54 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3560757164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:54 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3560757164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:55.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:54 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3560757164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:55.201 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:55.577 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:56.009 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:55 smithi050 ceph-mon[30378]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:56.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:55 smithi142 ceph-mon[35329]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:56.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:55 smithi082 ceph-mon[35046]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:26:56.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 -- 172.21.15.50:0/584629804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19381047e0 msgr2=0x7f1938108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 --2- 172.21.15.50:0/584629804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19381047e0 0x7f1938108830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a410 comp rx=0 tx=0).stop 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 -- 172.21.15.50:0/584629804 shutdown_connections 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 --2- 172.21.15.50:0/584629804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938108f60 0x7f1938110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 --2- 172.21.15.50:0/584629804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19381047e0 0x7f1938108830 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 --2- 172.21.15.50:0/584629804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1938103e30 0x7f1938104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 -- 172.21.15.50:0/584629804 >> 172.21.15.50:0/584629804 conn(0x7f19380ff910 msgr2=0x7f1938101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 -- 172.21.15.50:0/584629804 shutdown_connections 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.121+0000 7f1937200640 1 -- 172.21.15.50:0/584629804 wait complete. 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.122+0000 7f1937200640 1 Processor -- start 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 -- start start 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938103e30 0x7f193819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 0x7f193819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1938108f60 0x7f193819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1938113fc0 con 0x7f1938108f60 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1938113e40 con 0x7f1938103e30 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1937200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1938114140 con 0x7f19381047e0 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1935400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 0x7f193819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1935400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 0x7f193819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:33288/0 (socket says 172.21.15.50:33288) 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1935e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938103e30 0x7f193819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1935400640 1 -- 172.21.15.50:0/1799085816 learned_addr learned my addr 172.21.15.50:0/1799085816 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.123+0000 7f1936800640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1938108f60 0x7f193819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935400640 1 -- 172.21.15.50:0/1799085816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938103e30 msgr2=0x7f193819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935400640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938103e30 0x7f193819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935400640 1 -- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1938108f60 msgr2=0x7f193819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935400640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1938108f60 0x7f193819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935400640 1 -- 172.21.15.50:0/1799085816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819f460 con 0x7f19381047e0 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935e00640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938103e30 0x7f193819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f1935400640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 0x7f193819a9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1924006010 tx=0x7f1924004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.124+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192403f030 con 0x7f19381047e0 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.125+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819f6f0 con 0x7f19381047e0 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.125+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924004440 con 0x7f19381047e0 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.125+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924035070 con 0x7f19381047e0 2024-09-12T03:26:56.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.125+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381a7030 con 0x7f19381047e0 2024-09-12T03:26:56.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.126+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1938105310 con 0x7f19381047e0 2024-09-12T03:26:56.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.128+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1924042030 con 0x7f19381047e0 2024-09-12T03:26:56.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.129+0000 7f192ea00640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:56.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.129+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f19240efa00 con 0x7f19381047e0 2024-09-12T03:26:56.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.129+0000 7f1935e00640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:56.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.130+0000 7f1935e00640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f191c0a2b80 0x7f191c0a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1928009fc0 tx=0x7f1928005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:56.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.131+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19240bbf00 con 0x7f19381047e0 2024-09-12T03:26:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.355+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193810c140 con 0x7f19381047e0 2024-09-12T03:26:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.356+0000 7f192ea00640 1 -- 172.21.15.50:0/1799085816 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f19240c0200 con 0x7f19381047e0 2024-09-12T03:26:56.357 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:56.357 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"btime":"2024-09-12T03:25:24:859723+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:25:24.725697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/809625574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":809625574},{"type":"v1","addr":"172.21.15.142:6835","nonce":809625574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24508,24526,24536]},"id":1}]} 2024-09-12T03:26:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 25 2024-09-12T03:26:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f191c0a2b80 msgr2=0x7f191c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f191c0a2b80 0x7f191c0a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1928009fc0 tx=0x7f1928005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 msgr2=0x7f193819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 0x7f193819a9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1924006010 tx=0x7f1924004040 comp rx=0 tx=0).stop 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 shutdown_connections 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1938108f60 0x7f193819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19381047e0 0x7f193819a9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 --2- 172.21.15.50:0/1799085816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1938103e30 0x7f193819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 >> 172.21.15.50:0/1799085816 conn(0x7f19380ff910 msgr2=0x7f1938100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 shutdown_connections 2024-09-12T03:26:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:56.359+0000 7f1937200640 1 -- 172.21.15.50:0/1799085816 wait complete. 2024-09-12T03:26:57.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:56 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1799085816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:57.056 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:56 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1799085816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:57.092 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:26:57.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:56 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1799085816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:57.902 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:57 smithi050 ceph-mon[30378]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 -- 172.21.15.50:0/1854422110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 msgr2=0x7fa628107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 --2- 172.21.15.50:0/1854422110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628107da0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009930 tx=0x7fa61c02a420 comp rx=0 tx=0).stop 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 -- 172.21.15.50:0/1854422110 shutdown_connections 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 --2- 172.21.15.50:0/1854422110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa628102360 0x7fa62810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 --2- 172.21.15.50:0/1854422110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 0x7fa628101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 --2- 172.21.15.50:0/1854422110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628107da0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 -- 172.21.15.50:0/1854422110 >> 172.21.15.50:0/1854422110 conn(0x7fa6280fd6e0 msgr2=0x7fa6280ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:58.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 -- 172.21.15.50:0/1854422110 shutdown_connections 2024-09-12T03:26:58.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.014+0000 7fa62e800640 1 -- 172.21.15.50:0/1854422110 wait complete. 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 Processor -- start 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 -- start start 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 0x7fa6280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa628102360 0x7fa628074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa628111dc0 con 0x7fa6281079c0 2024-09-12T03:26:58.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa628111c40 con 0x7fa6281019c0 2024-09-12T03:26:58.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.015+0000 7fa62e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa628111f40 con 0x7fa628102360 2024-09-12T03:26:58.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:58.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53222/0 (socket says 172.21.15.50:53222) 2024-09-12T03:26:58.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 -- 172.21.15.50:0/2339198448 learned_addr learned my addr 172.21.15.50:0/2339198448 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:58.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62d400640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 0x7fa6280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:58.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62ca00640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa628102360 0x7fa628074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:58.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 -- 172.21.15.50:0/2339198448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa628102360 msgr2=0x7fa628074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa628102360 0x7fa628074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 -- 172.21.15.50:0/2339198448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 msgr2=0x7fa6280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 0x7fa6280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62de00640 1 -- 172.21.15.50:0/2339198448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6281a6780 con 0x7fa6281079c0 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.016+0000 7fa62d400640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 0x7fa6280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.017+0000 7fa62de00640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628074f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa62000eb10 tx=0x7fa62000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.017+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620004b20 con 0x7fa6281079c0 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.017+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa620004cc0 con 0x7fa6281079c0 2024-09-12T03:26:58.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.017+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620010760 con 0x7fa6281079c0 2024-09-12T03:26:58.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.017+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6281a6a70 con 0x7fa6281079c0 2024-09-12T03:26:58.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.017+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a6f60 con 0x7fa6281079c0 2024-09-12T03:26:58.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.018+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa628101e20 con 0x7fa6281079c0 2024-09-12T03:26:58.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.020+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa620005020 con 0x7fa6281079c0 2024-09-12T03:26:58.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.021+0000 7fa626000640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6100a2c00 0x7fa6100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:58.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.021+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fa6200cfad0 con 0x7fa6281079c0 2024-09-12T03:26:58.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.022+0000 7fa62d400640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6100a2c00 0x7fa6100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:58.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.022+0000 7fa62d400640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6100a2c00 0x7fa6100a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa61c000c00 tx=0x7fa61c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:58.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.026+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa62009c620 con 0x7fa6281079c0 2024-09-12T03:26:58.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:57 smithi142 ceph-mon[35329]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:26:58.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:57 smithi082 ceph-mon[35046]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:26:58.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.212+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa628075a20 con 0x7fa6281079c0 2024-09-12T03:26:58.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.214+0000 7fa626000640 1 -- 172.21.15.50:0/2339198448 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24820 (secure 0 0 0) 0x7fa628075a20 con 0x7fa6281079c0 2024-09-12T03:26:58.218 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:26:58.221 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":94,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:26:14.656446+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:26:58.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6100a2c00 msgr2=0x7fa6100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6100a2c00 0x7fa6100a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa61c000c00 tx=0x7fa61c005a20 comp rx=0 tx=0).stop 2024-09-12T03:26:58.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 msgr2=0x7fa628074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:58.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628074f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa62000eb10 tx=0x7fa62000c2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 shutdown_connections 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fa6100a2c00 0x7fa6100a50c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281079c0 0x7fa628074f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa628102360 0x7fa628074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 --2- 172.21.15.50:0/2339198448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6281019c0 0x7fa6280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 >> 172.21.15.50:0/2339198448 conn(0x7fa6280fd6e0 msgr2=0x7fa6280fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 shutdown_connections 2024-09-12T03:26:58.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:58.216+0000 7fa62e800640 1 -- 172.21.15.50:0/2339198448 wait complete. 2024-09-12T03:26:58.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:58 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2339198448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:58.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:24:32.635781+0000', 'modified': '2024-09-12T03:25:24.725697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24526, 'mds_2': 24508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/1268137141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 1268137141}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 1268137141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/4279679346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 4279679346}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 4279679346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/809625574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 809625574}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 809625574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.142:6837/2093449893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2093449893}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2093449893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.142:6839/3589126725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3589126725}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3589126725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.82:6839/2348373979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2348373979}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2348373979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24508, 24526, 24536]} 2024-09-12T03:26:58.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:26:58.920 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T03:26:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:58 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2339198448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:59.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2339198448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:59.296 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:59.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 -- 172.21.15.50:0/2434175164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80081047e0 msgr2=0x7f8008108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 --2- 172.21.15.50:0/2434175164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80081047e0 0x7f8008108830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009900 tx=0x7f7ff002a400 comp rx=0 tx=0).stop 2024-09-12T03:26:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 -- 172.21.15.50:0/2434175164 shutdown_connections 2024-09-12T03:26:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 --2- 172.21.15.50:0/2434175164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008108f60 0x7f8008110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 --2- 172.21.15.50:0/2434175164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80081047e0 0x7f8008108830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 --2- 172.21.15.50:0/2434175164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 0x7f8008104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 -- 172.21.15.50:0/2434175164 >> 172.21.15.50:0/2434175164 conn(0x7f80080ff8d0 msgr2=0x7f8008101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 -- 172.21.15.50:0/2434175164 shutdown_connections 2024-09-12T03:26:59.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.825+0000 7f800ce00640 1 -- 172.21.15.50:0/2434175164 wait complete. 2024-09-12T03:26:59.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.826+0000 7f800ce00640 1 Processor -- start 2024-09-12T03:26:59.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 -- start start 2024-09-12T03:26:59.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 0x7f800819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80081047e0 0x7f800819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 0x7f800819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8008113fd0 con 0x7f8008108f60 2024-09-12T03:26:59.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8008113e50 con 0x7f8008103e30 2024-09-12T03:26:59.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f800ce00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8008114150 con 0x7f80081047e0 2024-09-12T03:26:59.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f8007e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 0x7f800819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f8007400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 0x7f800819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f8007e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 0x7f800819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44332/0 (socket says 172.21.15.50:44332) 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f8007e00640 1 -- 172.21.15.50:0/3258408357 learned_addr learned my addr 172.21.15.50:0/3258408357 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.827+0000 7f7fffe00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80081047e0 0x7f800819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007e00640 1 -- 172.21.15.50:0/3258408357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80081047e0 msgr2=0x7f800819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007e00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80081047e0 0x7f800819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007e00640 1 -- 172.21.15.50:0/3258408357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 msgr2=0x7f800819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007e00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 0x7f800819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007e00640 1 -- 172.21.15.50:0/3258408357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800819f460 con 0x7f8008108f60 2024-09-12T03:26:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007400640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 0x7f800819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8007e00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 0x7f800819ed80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f800000cce0 tx=0x7f800000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8004c00640 1 -- 172.21.15.50:0/3258408357 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000f450 con 0x7f8008108f60 2024-09-12T03:26:59.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.828+0000 7f8004c00640 1 -- 172.21.15.50:0/3258408357 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800000fad0 con 0x7f8008108f60 2024-09-12T03:26:59.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.829+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800819f750 con 0x7f8008108f60 2024-09-12T03:26:59.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.829+0000 7f8004c00640 1 -- 172.21.15.50:0/3258408357 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000018c10 con 0x7f8008108f60 2024-09-12T03:26:59.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.829+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a7030 con 0x7f8008108f60 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.832+0000 7f8004c00640 1 -- 172.21.15.50:0/3258408357 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f800000f5f0 con 0x7f8008108f60 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.832+0000 7f8010dc0740 1 Processor -- start 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.832+0000 7f8010dc0740 1 -- start start 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.832+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ec88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.832+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.832+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8010dc0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563d78e2e0b0 con 0x563d78ec8e00 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8010dc0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563d78dd5ab0 con 0x563d78ed36c0 2024-09-12T03:26:59.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8010dc0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d78e2a070 con 0x563d78ec84e0 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f7fffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ec88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44338/0 (socket says 172.21.15.50:44338) 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 -- 172.21.15.50:0/3819509987 learned_addr learned my addr 172.21.15.50:0/3819509987 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f7fffe00640 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 -- 172.21.15.50:0/3819509987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 msgr2=0x563d78ec88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ec88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 -- 172.21.15.50:0/3819509987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 msgr2=0x563d78ed5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.833+0000 7f8007400640 1 -- 172.21.15.50:0/3819509987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d78de24a0 con 0x563d78ec8e00 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f8004c00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fec0a2c00 0x7f7fec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f8007400640 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed3180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8010480 tx=0x7f7ff8010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71100351e2ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f8004c00640 1 -- 172.21.15.50:0/3258408357 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f80000cf970 con 0x7f8008108f60 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f7fffe00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fec0a2c00 0x7f7fec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f7ffcc00640 1 -- 172.21.15.50:0/3819509987 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8011760 con 0x563d78ec8e00 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f7ffcc00640 1 -- 172.21.15.50:0/3819509987 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8011de0 con 0x563d78ec8e00 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.834+0000 7f7ffcc00640 1 -- 172.21.15.50:0/3819509987 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80193d0 con 0x563d78ec8e00 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 -- 172.21.15.50:0/3819509987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 msgr2=0x563d78ed3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed3180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8010480 tx=0x7f7ff8010940 comp rx=0 tx=0).stop 2024-09-12T03:26:59.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f7fffe00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fec0a2c00 0x7f7fec0a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000c00 tx=0x7f7ff0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 -- 172.21.15.50:0/3819509987 shutdown_connections 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed3180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 --2- 172.21.15.50:0/3819509987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ec88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 -- 172.21.15.50:0/3819509987 >> 172.21.15.50:0/3819509987 conn(0x563d78ec4a70 msgr2=0x563d78ec4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 -- 172.21.15.50:0/3819509987 shutdown_connections 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.835+0000 7f8010dc0740 1 -- 172.21.15.50:0/3819509987 wait complete. 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 -- start start 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 Processor -- start 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ed05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563d78ed6330 con 0x563d78ec8e00 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563d78db6920 con 0x563d78ed36c0 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.836+0000 7f8010dc0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d78ed64b0 con 0x563d78ec84e0 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8010dc0740 10 client.0 _ll_register_callbacks cb 0x563d78dda8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f7fffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ed05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f7fffe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44352/0 (socket says 172.21.15.50:44352) 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f7fffe00640 1 -- 172.21.15.50:0/568217784 learned_addr learned my addr 172.21.15.50:0/568217784 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007e00640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007400640 1 -- 172.21.15.50:0/568217784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 msgr2=0x563d78ed1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007400640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007400640 1 -- 172.21.15.50:0/568217784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 msgr2=0x563d78ed0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007400640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f8007400640 1 -- 172.21.15.50:0/568217784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d78b9f2b0 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.837+0000 7f7fffe00640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f8007400640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ed05a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x563d78ec58d0 tx=0x7f7ff800e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74d0d848a3412b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f7ff5600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f7ff5600640 1 -- 172.21.15.50:0/568217784 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800e430 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff8011de0 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f7ff5600640 1 -- 172.21.15.50:0/568217784 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f7ff801f030 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f7ff5600640 1 -- 172.21.15.50:0/568217784 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ff8030050 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f8010dc0740 10 client.25087 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.838+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d78ee2030 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f7ff5600640 1 -- 172.21.15.50:0/568217784 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8011de0 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f7ff5600640 1 -- 172.21.15.50:0/568217784 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7ff80051d0 con 0x563d78ec84e0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f8010dc0740 10 client.25087 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f8010dc0740 10 client.25087 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f8010dc0740 20 client.25087 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f8010dc0740 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78ee4380 0x563d78ee57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f8010dc0740 4 client.25087 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 --> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d78e3b620 con 0x563d78ee4380 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.839+0000 7f7fffe00640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78ee4380 0x563d78ee57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.840+0000 7f7fffe00640 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78ee4380 0x563d78ee57f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff066853ff40fc02 server_cookie=d604c78d4f82836c in_seq=0 out_seq=0 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.840+0000 7f7ff5600640 10 client.25087 ms_handle_connect on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f7ff5600640 1 -- 172.21.15.50:0/568217784 <== mds.2 v2:172.21.15.142:6836/2093449893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d78e3b620 con 0x563d78ee4380 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f7ff5600640 10 client.25087 handle_command_reply: tid=0 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 client.25087 shutdown 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 msgr2=0x563d78ed05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ed05a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x563d78ec58d0 tx=0x7f7ff800e830 comp rx=0 tx=0).stop 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 shutdown_connections 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ed36c0 0x563d78ed1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78ee4380 0x563d78ee57f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec8e00 0x563d78ed0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 --2- 172.21.15.50:0/568217784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ec84e0 0x563d78ed05a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 >> 172.21.15.50:0/568217784 conn(0x563d78ec4a70 msgr2=0x563d78eca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f7ff5600640 0 client.25087 ms_handle_reset on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 shutdown_connections 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 1 -- 172.21.15.50:0/568217784 wait complete. 2024-09-12T03:26:59.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.841+0000 7f8010dc0740 20 client.25087 trim_cache size 0 max 16384 2024-09-12T03:26:59.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.860+0000 7f8010dc0740 1 Processor -- start 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.860+0000 7f8010dc0740 1 -- start start 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78ee5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78ed0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ecd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8010dc0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563d78ee3b30 con 0x563d78ec84e0 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8010dc0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563d78ee4200 con 0x563d78ec8e00 2024-09-12T03:26:59.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8010dc0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d78ee48d0 con 0x563d78ed36c0 2024-09-12T03:26:59.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78ee5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78ee5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44364/0 (socket says 172.21.15.50:44364) 2024-09-12T03:26:59.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8007400640 1 -- 172.21.15.50:0/906641750 learned_addr learned my addr 172.21.15.50:0/906641750 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:59.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f7fffe00640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78ed0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8007e00640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ecd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.861+0000 7f8007400640 1 -- 172.21.15.50:0/906641750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 msgr2=0x563d78ecd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f8007400640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ecd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f8007400640 1 -- 172.21.15.50:0/906641750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 msgr2=0x563d78ed0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f8007400640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78ed0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f8007400640 1 -- 172.21.15.50:0/906641750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d78ec8c30 con 0x563d78ec84e0 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f8007e00640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ecd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f7fffe00640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78ed0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f8007400640 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78ee5120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8010480 tx=0x7f7ff800bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589631a2f6d22395 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f7ff7e00640 1 -- 172.21.15.50:0/906641750 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8039070 con 0x563d78ec84e0 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f7ff7e00640 1 -- 172.21.15.50:0/906641750 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8011630 con 0x563d78ec84e0 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.862+0000 7f7ff7e00640 1 -- 172.21.15.50:0/906641750 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ebc0 con 0x563d78ec84e0 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 -- 172.21.15.50:0/906641750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 msgr2=0x563d78ee5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78ee5120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8010480 tx=0x7f7ff800bae0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 -- 172.21.15.50:0/906641750 shutdown_connections 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ecd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78ed0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 --2- 172.21.15.50:0/906641750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78ee5120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 -- 172.21.15.50:0/906641750 >> 172.21.15.50:0/906641750 conn(0x563d78ec4a70 msgr2=0x563d78ecee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 -- 172.21.15.50:0/906641750 shutdown_connections 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.863+0000 7f8010dc0740 1 -- 172.21.15.50:0/906641750 wait complete. 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 -- start start 2024-09-12T03:26:59.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 Processor -- start 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78eeb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78eebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ee6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563d78ee5670 con 0x563d78ec84e0 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563d78ee4fa0 con 0x563d78ec8e00 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d78ee5d40 con 0x563d78ed36c0 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.864+0000 7f8010dc0740 10 client.0 _ll_register_callbacks cb 0x7f7ff800bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:59.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78eeb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78eeb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44376/0 (socket says 172.21.15.50:44376) 2024-09-12T03:26:59.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007400640 1 -- 172.21.15.50:0/767516669 learned_addr learned my addr 172.21.15.50:0/767516669 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:26:59.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ee6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f7fffe00640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78eebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 -- 172.21.15.50:0/767516669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 msgr2=0x563d78eebde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78eebde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 -- 172.21.15.50:0/767516669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 msgr2=0x563d78eeb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78eeb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 -- 172.21.15.50:0/767516669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d78ee7460 con 0x563d78ed36c0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007400640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78eeb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f8007e00640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ee6e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f80000cd030 tx=0x7f800009c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32c7a010d4c8c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.865+0000 7f7ff5600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f7ff5600640 1 -- 172.21.15.50:0/767516669 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000b8020 con 0x563d78ed36c0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff8011630 con 0x563d78ed36c0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f7ff5600640 1 -- 172.21.15.50:0/767516669 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f80000a45a0 con 0x563d78ed36c0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f7ff5600640 1 -- 172.21.15.50:0/767516669 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8000083050 con 0x563d78ed36c0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f8010dc0740 10 client.25093 fetch_fsmap learned FSMap version 25 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d78eeafc0 con 0x563d78ed36c0 2024-09-12T03:26:59.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.866+0000 7f7ff5600640 1 -- 172.21.15.50:0/767516669 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8011630 con 0x563d78ed36c0 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f7ff5600640 1 -- 172.21.15.50:0/767516669 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f800009b330 con 0x563d78ed36c0 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8010dc0740 10 client.25093 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8010dc0740 10 client.25093 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8010dc0740 20 client.25093 populate_metadata read hostname 'smithi050' 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8010dc0740 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78f52fe0 0x563d78f51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8010dc0740 4 client.25093 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 --> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] -- command(tid 0: {"prefix": "status"}) -- 0x563d78e35b30 con 0x563d78f52fe0 2024-09-12T03:26:59.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.867+0000 7f8007400640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78f52fe0 0x563d78f51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8007400640 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78f52fe0 0x563d78f51cf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4e0e433bdcb07b3 server_cookie=db93caad1972d060 in_seq=0 out_seq=0 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f7ff5600640 10 client.25093 ms_handle_connect on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f7ff5600640 1 -- 172.21.15.50:0/767516669 <== mds.2 v2:172.21.15.142:6836/2093449893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x563d78e35b30 con 0x563d78f52fe0 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f7ff5600640 10 client.25093 handle_command_reply: tid=0 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 client.25093 shutdown 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 msgr2=0x563d78ee6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ee6e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f80000cd030 tx=0x7f800009c4c0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 shutdown_connections 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d78ed36c0 0x563d78ee6e20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d78ec8e00 0x563d78eebde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.142:6836/2093449893,v1:172.21.15.142:6837/2093449893] conn(0x563d78f52fe0 0x563d78f51cf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 --2- 172.21.15.50:0/767516669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d78ec84e0 0x563d78eeb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 >> 172.21.15.50:0/767516669 conn(0x563d78ec4a70 msgr2=0x563d78ed2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f7ff5600640 0 client.25093 ms_handle_reset on v2:172.21.15.142:6836/2093449893 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 shutdown_connections 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.868+0000 7f8010dc0740 1 -- 172.21.15.50:0/767516669 wait complete. 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f8010dc0740 20 client.25093 trim_cache size 0 max 16384 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:26:59.874 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24508, 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 98.01576627, 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 94, 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 174.863680614 2024-09-12T03:26:59.875 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fec0a2c00 msgr2=0x7f7fec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fec0a2c00 0x7f7fec0a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000c00 tx=0x7f7ff0009260 comp rx=0 tx=0).stop 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 msgr2=0x7f800819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 0x7f800819ed80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f800000cce0 tx=0x7f800000a620 comp rx=0 tx=0).stop 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 shutdown_connections 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f7fec0a2c00 0x7f7fec0a50c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8008108f60 0x7f800819ed80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80081047e0 0x7f800819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 --2- 172.21.15.50:0/3258408357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8008103e30 0x7f800819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 >> 172.21.15.50:0/3258408357 conn(0x7f80080ff8d0 msgr2=0x7f8008100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 shutdown_connections 2024-09-12T03:26:59.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:26:59.869+0000 7f800ce00640 1 -- 172.21.15.50:0/3258408357 wait complete. 2024-09-12T03:27:00.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:26:59 smithi082 ceph-mon[35046]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:27:00.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:26:59 smithi050 ceph-mon[30378]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:27:00.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:26:59 smithi142 ceph-mon[35329]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:27:00.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:27:00.972 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.523+0000 7f33f8000640 1 -- 172.21.15.50:0/1018720846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 msgr2=0x7f33f01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.523+0000 7f33f8000640 1 --2- 172.21.15.50:0/1018720846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f01089f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009900 tx=0x7f33dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 -- 172.21.15.50:0/1018720846 shutdown_connections 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 --2- 172.21.15.50:0/1018720846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0109120 0x7f33f0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 --2- 172.21.15.50:0/1018720846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f01089f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 --2- 172.21.15.50:0/1018720846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0103ff0 0x7f33f01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 -- 172.21.15.50:0/1018720846 >> 172.21.15.50:0/1018720846 conn(0x7f33f00ffa90 msgr2=0x7f33f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 -- 172.21.15.50:0/1018720846 shutdown_connections 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.524+0000 7f33f8000640 1 -- 172.21.15.50:0/1018720846 wait complete. 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 Processor -- start 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 -- start start 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ff0 0x7f33f019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109120 0x7f33f019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0114140 con 0x7f33f01049a0 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0113fc0 con 0x7f33f0103ff0 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f8000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01142c0 con 0x7f33f0109120 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.525+0000 7f33f4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44404/0 (socket says 172.21.15.50:44404) 2024-09-12T03:27:01.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f4c00640 1 -- 172.21.15.50:0/3344525786 learned_addr learned my addr 172.21.15.50:0/3344525786 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ff0 0x7f33f019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f6000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109120 0x7f33f019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 -- 172.21.15.50:0/3344525786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109120 msgr2=0x7f33f019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109120 0x7f33f019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 -- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 msgr2=0x7f33f019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 -- 172.21.15.50:0/3344525786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f019f5f0 con 0x7f33f0103ff0 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f4c00640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:01.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.526+0000 7f33f5600640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ff0 0x7f33f019a640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e9f0 tx=0x7f33e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.527+0000 7f33ee000640 1 -- 172.21.15.50:0/3344525786 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e400ccc0 con 0x7f33f0103ff0 2024-09-12T03:27:01.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.527+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f019f8e0 con 0x7f33f0103ff0 2024-09-12T03:27:01.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.527+0000 7f33ee000640 1 -- 172.21.15.50:0/3344525786 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e4005020 con 0x7f33f0103ff0 2024-09-12T03:27:01.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.527+0000 7f33ee000640 1 -- 172.21.15.50:0/3344525786 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e40049e0 con 0x7f33f0103ff0 2024-09-12T03:27:01.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.527+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f01a71c0 con 0x7f33f0103ff0 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 Processor -- start 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 -- start start 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33ee000640 1 -- 172.21.15.50:0/3344525786 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33e4005560 con 0x7f33f0103ff0 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff31710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff31c50 0x556dfff3bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff3c510 0x556dfff3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556dffe970b0 con 0x556dfff31c50 2024-09-12T03:27:01.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dffe3eab0 con 0x556dfff3c510 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.530+0000 7f33fac09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556dffe93070 con 0x556dfff31330 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff31c50 0x556dfff3bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff31710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff31c50 0x556dfff3bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44418/0 (socket says 172.21.15.50:44418) 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f5600640 1 -- 172.21.15.50:0/2713045089 learned_addr learned my addr 172.21.15.50:0/2713045089 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f4c00640 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff3c510 0x556dfff3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f6000640 1 -- 172.21.15.50:0/2713045089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff3c510 msgr2=0x556dfff3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f6000640 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff3c510 0x556dfff3e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f6000640 1 -- 172.21.15.50:0/2713045089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff31c50 msgr2=0x556dfff3bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f6000640 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff31c50 0x556dfff3bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.531+0000 7f33f6000640 1 -- 172.21.15.50:0/2713045089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dffe4b4a0 con 0x556dfff31330 2024-09-12T03:27:01.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33f6000640 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff31710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009eb0 tx=0x7f33e80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9cde189500c752 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33e2600640 1 -- 172.21.15.50:0/2713045089 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8003b70 con 0x556dfff31330 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33ee000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f33d80a2b80 0x7f33d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33e2600640 1 -- 172.21.15.50:0/2713045089 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e8015070 con 0x556dfff31330 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33e2600640 1 -- 172.21.15.50:0/2713045089 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e801a940 con 0x556dfff31330 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33ee000640 1 -- 172.21.15.50:0/3344525786 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f33e40cf3d0 con 0x7f33f0103ff0 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.532+0000 7f33f5600640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f33d80a2b80 0x7f33d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 -- 172.21.15.50:0/2713045089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 msgr2=0x556dfff31710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff31710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009eb0 tx=0x7f33e80107a0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 -- 172.21.15.50:0/2713045089 shutdown_connections 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff3c510 0x556dfff3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff31c50 0x556dfff3bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 --2- 172.21.15.50:0/2713045089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff31710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 -- 172.21.15.50:0/2713045089 >> 172.21.15.50:0/2713045089 conn(0x556dfff2d8c0 msgr2=0x556dfff2dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33f5600640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f33d80a2b80 0x7f33d80a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e9c0 tx=0x7f33e40d2e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 -- 172.21.15.50:0/2713045089 shutdown_connections 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 -- 172.21.15.50:0/2713045089 wait complete. 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.533+0000 7f33fac09740 1 -- start start 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 Processor -- start 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff39ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556dfff3f060 con 0x556dfff3c510 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dffe1f920 con 0x556dfff31c50 2024-09-12T03:27:01.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556dfff3f1e0 con 0x556dfff31330 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33fac09740 10 client.0 _ll_register_callbacks cb 0x556dffe438a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33f6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff39ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.534+0000 7f33f6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58040/0 (socket says 172.21.15.50:58040) 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 -- 172.21.15.50:0/578570252 learned_addr learned my addr 172.21.15.50:0/578570252 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 -- 172.21.15.50:0/578570252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 msgr2=0x556dfff39ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff39ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 -- 172.21.15.50:0/578570252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 msgr2=0x556dfff39ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff39ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 -- 172.21.15.50:0/578570252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dffc082b0 con 0x556dfff31330 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33f6000640 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff39570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33e80047d0 tx=0x7f33e8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c6241f25c61b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.535+0000 7f33d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:27:01.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33d6a00640 1 -- 172.21.15.50:0/578570252 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8023040 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e8015070 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33d6a00640 1 -- 172.21.15.50:0/578570252 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f33e8004c40 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33d6a00640 1 -- 172.21.15.50:0/578570252 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33e8039050 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33fac09740 10 client.25105 fetch_fsmap learned FSMap version 25 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556dfff4adf0 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33d6a00640 1 -- 172.21.15.50:0/578570252 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e8015070 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.536+0000 7f33d6a00640 1 -- 172.21.15.50:0/578570252 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33e8004050 con 0x556dfff31330 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33fac09740 10 client.25105 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33fac09740 10 client.25105 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33fac09740 20 client.25105 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33fac09740 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfff4d1d0 0x556dfff4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33fac09740 4 client.25105 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:01.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 --> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dffea4620 con 0x556dfff4d1d0 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.537+0000 7f33f4c00640 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfff4d1d0 0x556dfff4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.538+0000 7f33f4c00640 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfff4d1d0 0x556dfff4e640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6764390fac2d1f47 server_cookie=851ef01ccfd251d4 in_seq=0 out_seq=0 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.538+0000 7f33d6a00640 10 client.25105 ms_handle_connect on v2:172.21.15.142:6838/3589126725 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.538+0000 7f33d6a00640 1 -- 172.21.15.50:0/578570252 <== mds.1 v2:172.21.15.142:6838/3589126725 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556dffea4620 con 0x556dfff4d1d0 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.538+0000 7f33d6a00640 10 client.25105 handle_command_reply: tid=0 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 client.25105 shutdown 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 msgr2=0x556dfff39570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff39570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33e80047d0 tx=0x7f33e8000f40 comp rx=0 tx=0).stop 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 shutdown_connections 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfff4d1d0 0x556dfff4e640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 --2- 172.21.15.50:0/578570252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff39570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 >> 172.21.15.50:0/578570252 conn(0x556dfff2d8c0 msgr2=0x556dfff335f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33d6a00640 0 client.25105 ms_handle_reset on v2:172.21.15.142:6838/3589126725 2024-09-12T03:27:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 shutdown_connections 2024-09-12T03:27:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 1 -- 172.21.15.50:0/578570252 wait complete. 2024-09-12T03:27:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.539+0000 7f33fac09740 20 client.25105 trim_cache size 0 max 16384 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 Processor -- start 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 -- start start 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff33de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556dfff4c980 con 0x556dfff3c510 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dfff4d050 con 0x556dfff31c50 2024-09-12T03:27:01.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.558+0000 7f33fac09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556dfff4d720 con 0x556dfff31330 2024-09-12T03:27:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff33de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44440/0 (socket says 172.21.15.50:44440) 2024-09-12T03:27:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f6000640 1 -- 172.21.15.50:0/4025401551 learned_addr learned my addr 172.21.15.50:0/4025401551 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:01.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f6000640 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 -- 172.21.15.50:0/4025401551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 msgr2=0x556dfff4cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff4cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 -- 172.21.15.50:0/4025401551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 msgr2=0x556dfff36890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 -- 172.21.15.50:0/4025401551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfff2d6e0 con 0x556dfff31c50 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f6000640 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.559+0000 7f33f4c00640 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff33de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009230 tx=0x7f33dc005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f55ff4b509267b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33e1c00640 1 -- 172.21.15.50:0/4025401551 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc035040 con 0x556dfff31c50 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33e1c00640 1 -- 172.21.15.50:0/4025401551 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc04a070 con 0x556dfff31c50 2024-09-12T03:27:01.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33e1c00640 1 -- 172.21.15.50:0/4025401551 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0049c0 con 0x556dfff31c50 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 -- 172.21.15.50:0/4025401551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 msgr2=0x556dfff33de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff33de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009230 tx=0x7f33dc005a20 comp rx=0 tx=0).stop 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 -- 172.21.15.50:0/4025401551 shutdown_connections 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff33de0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 --2- 172.21.15.50:0/4025401551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff4cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 -- 172.21.15.50:0/4025401551 >> 172.21.15.50:0/4025401551 conn(0x556dfff2d8c0 msgr2=0x556dfff37ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 -- 172.21.15.50:0/4025401551 shutdown_connections 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.560+0000 7f33fac09740 1 -- 172.21.15.50:0/4025401551 wait complete. 2024-09-12T03:27:01.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 -- start start 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 Processor -- start 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff57130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff36420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556dfff4e4c0 con 0x556dfff3c510 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dfff4ddf0 con 0x556dfff31c50 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556dfff4eb90 con 0x556dfff31330 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.561+0000 7f33fac09740 10 client.0 _ll_register_callbacks cb 0x7f33e801a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff57130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff36420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:55848/0 (socket says 172.21.15.50:55848) 2024-09-12T03:27:01.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f4c00640 1 -- 172.21.15.50:0/3523860715 learned_addr learned my addr 172.21.15.50:0/3523860715 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 -- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 msgr2=0x556dfff57130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff57130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 -- 172.21.15.50:0/3523860715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 msgr2=0x556dfff36420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff36420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 -- 172.21.15.50:0/3523860715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfff37250 con 0x556dfff3c510 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f4c00640 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff36420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33f6000640 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36a50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f33e8002230 tx=0x7f33e8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3704beed58f422 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.562+0000 7f33d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:27:01.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33d6a00640 1 -- 172.21.15.50:0/3523860715 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8010d10 con 0x556dfff3c510 2024-09-12T03:27:01.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33d6a00640 1 -- 172.21.15.50:0/3523860715 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f33e802be00 con 0x556dfff3c510 2024-09-12T03:27:01.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33dc04a070 con 0x556dfff3c510 2024-09-12T03:27:01.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33d6a00640 1 -- 172.21.15.50:0/3523860715 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33e8003970 con 0x556dfff3c510 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33fac09740 10 client.15096 fetch_fsmap learned FSMap version 25 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556dfffb79d0 con 0x556dfff3c510 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33d6a00640 1 -- 172.21.15.50:0/3523860715 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc04a070 con 0x556dfff3c510 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.563+0000 7f33d6a00640 1 -- 172.21.15.50:0/3523860715 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33e8035070 con 0x556dfff3c510 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33fac09740 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33fac09740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33fac09740 20 client.15096 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33fac09740 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfffbc560 0x556dfffbb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33fac09740 4 client.15096 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 --> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] -- command(tid 0: {"prefix": "status"}) -- 0x556dffe9eb30 con 0x556dfffbc560 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.564+0000 7f33f4c00640 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfffbc560 0x556dfffbb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33f4c00640 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfffbc560 0x556dfffbb270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d53ceb00cfe0dc82 server_cookie=ac054531ebf15418 in_seq=0 out_seq=0 2024-09-12T03:27:01.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33d6a00640 10 client.15096 ms_handle_connect on v2:172.21.15.142:6838/3589126725 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33d6a00640 1 -- 172.21.15.50:0/3523860715 <== mds.1 v2:172.21.15.142:6838/3589126725 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556dffe9eb30 con 0x556dfffbc560 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33d6a00640 10 client.15096 handle_command_reply: tid=0 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 client.15096 shutdown 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 msgr2=0x556dfff36a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36a50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f33e8002230 tx=0x7f33e8013040 comp rx=0 tx=0).stop 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 shutdown_connections 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:6838/3589126725,v1:172.21.15.142:6839/3589126725] conn(0x556dfffbc560 0x556dfffbb270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dfff3c510 0x556dfff36a50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfff31c50 0x556dfff36420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 --2- 172.21.15.50:0/3523860715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556dfff31330 0x556dfff57130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.565+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 >> 172.21.15.50:0/3523860715 conn(0x556dfff2d8c0 msgr2=0x556dfff47430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33d6a00640 0 client.15096 ms_handle_reset on v2:172.21.15.142:6838/3589126725 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 shutdown_connections 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33fac09740 1 -- 172.21.15.50:0/3523860715 wait complete. 2024-09-12T03:27:01.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33fac09740 20 client.15096 trim_cache size 0 max 16384 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24526, 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 100.71751881100001, 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-09-12T03:27:01.570 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 94, 2024-09-12T03:27:01.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:27:01.571 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 171.23798848600001 2024-09-12T03:27:01.571 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:27:01.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f33d80a2b80 msgr2=0x7f33d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33f8000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f33d80a2b80 0x7f33d80a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e9c0 tx=0x7f33e40d2e10 comp rx=0 tx=0).stop 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ff0 msgr2=0x7f33f019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.566+0000 7f33f8000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ff0 0x7f33f019a640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e9f0 tx=0x7f33e400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 shutdown_connections 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f33d80a2b80 0x7f33d80a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33f0109120 0x7f33f019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33f01049a0 0x7f33f019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 --2- 172.21.15.50:0/3344525786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ff0 0x7f33f019a640 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 >> 172.21.15.50:0/3344525786 conn(0x7f33f00ffa90 msgr2=0x7f33f0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 shutdown_connections 2024-09-12T03:27:01.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:01.567+0000 7f33f8000640 1 -- 172.21.15.50:0/3344525786 wait complete. 2024-09-12T03:27:02.079 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:01 smithi050 ceph-mon[30378]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:27:02.096 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:01 smithi142 ceph-mon[35329]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:27:02.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:01 smithi082 ceph-mon[35046]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:27:02.444 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:27:02.821 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 -- 172.21.15.50:0/3093299672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074620 msgr2=0x7fe6c0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/3093299672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074620 0x7fe6c0074a00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009900 tx=0x7fe6b402a480 comp rx=0 tx=0).stop 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 -- 172.21.15.50:0/3093299672 shutdown_connections 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/3093299672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c010e3c0 0x7fe6c0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/3093299672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c0074f40 0x7fe6c010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/3093299672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074620 0x7fe6c0074a00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 -- 172.21.15.50:0/3093299672 >> 172.21.15.50:0/3093299672 conn(0x7fe6c00fd8c0 msgr2=0x7fe6c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 -- 172.21.15.50:0/3093299672 shutdown_connections 2024-09-12T03:27:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.349+0000 7fe6c7a00640 1 -- 172.21.15.50:0/3093299672 wait complete. 2024-09-12T03:27:03.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 Processor -- start 2024-09-12T03:27:03.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 -- start start 2024-09-12T03:27:03.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 0x7fe6c019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074f40 0x7fe6c019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c010e3c0 0x7fe6c019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01141a0 con 0x7fe6c0074f40 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0114020 con 0x7fe6c0074620 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.350+0000 7fe6c7a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0114320 con 0x7fe6c010e3c0 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074f40 0x7fe6c019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 0x7fe6c019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 0x7fe6c019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:55870/0 (socket says 172.21.15.50:55870) 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 -- 172.21.15.50:0/1165666487 learned_addr learned my addr 172.21.15.50:0/1165666487 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:03.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c010e3c0 0x7fe6c019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 -- 172.21.15.50:0/1165666487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c010e3c0 msgr2=0x7fe6c019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c010e3c0 0x7fe6c019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 -- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074f40 msgr2=0x7fe6c019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074f40 0x7fe6c019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.351+0000 7fe6c5000640 1 -- 172.21.15.50:0/1165666487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c019f630 con 0x7fe6c0074620 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.352+0000 7fe6c5000640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 0x7fe6c019a680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4004770 tx=0x7fe6b4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.352+0000 7fe6bd600640 1 -- 172.21.15.50:0/1165666487 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b403f070 con 0x7fe6c0074620 2024-09-12T03:27:03.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.352+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c019f8c0 con 0x7fe6c0074620 2024-09-12T03:27:03.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.352+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a7200 con 0x7fe6c0074620 2024-09-12T03:27:03.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.352+0000 7fe6bd600640 1 -- 172.21.15.50:0/1165666487 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4031ca0 con 0x7fe6c0074620 2024-09-12T03:27:03.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.352+0000 7fe6bd600640 1 -- 172.21.15.50:0/1165666487 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4049060 con 0x7fe6c0074620 2024-09-12T03:27:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.355+0000 7fe6ca3c1740 1 Processor -- start 2024-09-12T03:27:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.355+0000 7fe6ca3c1740 1 -- start start 2024-09-12T03:27:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.355+0000 7fe6bd600640 1 -- 172.21.15.50:0/1165666487 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe6b4042070 con 0x7fe6c0074620 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dcdd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dcea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56521dc430b0 con 0x56521dcddc50 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56521dbeaab0 con 0x56521dce8510 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56521dc3f070 con 0x56521dcdd330 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6bfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dcea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44484/0 (socket says 172.21.15.50:44484) 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6c5000640 1 -- 172.21.15.50:0/2842088010 learned_addr learned my addr 172.21.15.50:0/2842088010 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.356+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dcdd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bfe00640 1 -- 172.21.15.50:0/2842088010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 msgr2=0x56521dcdd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dcdd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bfe00640 1 -- 172.21.15.50:0/2842088010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 msgr2=0x56521dce7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bfe00640 1 -- 172.21.15.50:0/2842088010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521dbf74a0 con 0x56521dce8510 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dcdd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dcea900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac005d00 tx=0x7fe6ac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b6a4d975290252 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bd600640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe6a80a2b80 0x7fe6a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6b2000640 1 -- 172.21.15.50:0/2842088010 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac01c480 con 0x56521dce8510 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6bd600640 1 -- 172.21.15.50:0/1165666487 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe6b40ef830 con 0x7fe6c0074620 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6b2000640 1 -- 172.21.15.50:0/2842088010 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac01cb00 con 0x56521dce8510 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6b2000640 1 -- 172.21.15.50:0/2842088010 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac011b80 con 0x56521dce8510 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.357+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe6a80a2b80 0x7fe6a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/2842088010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 msgr2=0x56521dcea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dcea900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac005d00 tx=0x7fe6ac0107a0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe6a80a2b80 0x7fe6a80a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8005fd0 tx=0x7fe6b80096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/2842088010 shutdown_connections 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dcea900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/2842088010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dcdd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/2842088010 >> 172.21.15.50:0/2842088010 conn(0x56521dcd98c0 msgr2=0x56521dcd9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.358+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/2842088010 shutdown_connections 2024-09-12T03:27:03.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.359+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/2842088010 wait complete. 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.359+0000 7fe6ca3c1740 1 -- start start 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.359+0000 7fe6ca3c1740 1 Processor -- start 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dce5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56521dceb060 con 0x56521dcddc50 2024-09-12T03:27:03.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56521dbcb920 con 0x56521dce8510 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56521dceb1e0 con 0x56521dcdd330 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6ca3c1740 10 client.0 _ll_register_callbacks cb 0x56521dbef8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dce5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dce5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58092/0 (socket says 172.21.15.50:58092) 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 -- 172.21.15.50:0/506537390 learned_addr learned my addr 172.21.15.50:0/506537390 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 -- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 msgr2=0x56521dce6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 -- 172.21.15.50:0/506537390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 msgr2=0x56521dce5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.360+0000 7fe6c5000640 1 -- 172.21.15.50:0/506537390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521d9b42b0 con 0x56521dcdd330 2024-09-12T03:27:03.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6c5000640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dce5610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4036040 tx=0x7fe6b403b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dabb9826b9196121 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6a6a00640 1 -- 172.21.15.50:0/506537390 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40bb870 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ac01cb00 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6a6a00640 1 -- 172.21.15.50:0/506537390 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe6b403b5f0 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6a6a00640 1 -- 172.21.15.50:0/506537390 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6b40bb690 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6ca3c1740 10 client.25123 fetch_fsmap learned FSMap version 25 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.361+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56521dceb110 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6a6a00640 1 -- 172.21.15.50:0/506537390 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac01cb00 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6a6a00640 1 -- 172.21.15.50:0/506537390 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe6b404c060 con 0x56521dcdd330 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6ca3c1740 10 client.25123 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6ca3c1740 10 client.25123 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6ca3c1740 20 client.25123 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:03.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dcf9540 0x56521dcfa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6ca3c1740 4 client.25123 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:03.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56521dc50620 con 0x56521dcf9540 2024-09-12T03:27:03.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.362+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dcf9540 0x56521dcfa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.363+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dcf9540 0x56521dcfa9b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a677967b5256f4 server_cookie=5711cedfc803cca9 in_seq=0 out_seq=0 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.363+0000 7fe6a6a00640 10 client.25123 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6a6a00640 1 -- 172.21.15.50:0/506537390 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56521dc50620 con 0x56521dcf9540 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6a6a00640 10 client.25123 handle_command_reply: tid=0 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 client.25123 shutdown 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 msgr2=0x56521dce5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dce5610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4036040 tx=0x7fe6b403b520 comp rx=0 tx=0).stop 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 shutdown_connections 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcddc50 0x56521dce5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dcf9540 0x56521dcfa9b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/506537390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcdd330 0x56521dce5610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 >> 172.21.15.50:0/506537390 conn(0x56521dcd98c0 msgr2=0x56521dcdf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6a6a00640 0 client.25123 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.364+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 shutdown_connections 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.365+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/506537390 wait complete. 2024-09-12T03:27:03.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.365+0000 7fe6ca3c1740 20 client.25123 trim_cache size 0 max 16384 2024-09-12T03:27:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.383+0000 7fe6ca3c1740 1 Processor -- start 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.383+0000 7fe6ca3c1740 1 -- start start 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dce0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcddc50 0x56521dcdfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56521dcf8cf0 con 0x56521dcdd330 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56521dcf93c0 con 0x56521dce8510 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56521dcf9a90 con 0x56521dcddc50 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dce0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dce0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44502/0 (socket says 172.21.15.50:44502) 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 -- 172.21.15.50:0/3190181344 learned_addr learned my addr 172.21.15.50:0/3190181344 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcddc50 0x56521dcdfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 -- 172.21.15.50:0/3190181344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcddc50 msgr2=0x56521dcdfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcddc50 0x56521dcdfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 -- 172.21.15.50:0/3190181344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 msgr2=0x56521dce5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.384+0000 7fe6c5000640 1 -- 172.21.15.50:0/3190181344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521dcdda80 con 0x56521dcdd330 2024-09-12T03:27:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.385+0000 7fe6c5000640 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dce0930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40bbcc0 tx=0x7fe6b402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da53f69193752aff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.385+0000 7fe6b1600640 1 -- 172.21.15.50:0/3190181344 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40bbd30 con 0x56521dcdd330 2024-09-12T03:27:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.385+0000 7fe6b1600640 1 -- 172.21.15.50:0/3190181344 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40c0030 con 0x56521dcdd330 2024-09-12T03:27:03.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.385+0000 7fe6b1600640 1 -- 172.21.15.50:0/3190181344 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40bb3e0 con 0x56521dcdd330 2024-09-12T03:27:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/3190181344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 msgr2=0x56521dce0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dce0930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40bbcc0 tx=0x7fe6b402a480 comp rx=0 tx=0).stop 2024-09-12T03:27:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/3190181344 shutdown_connections 2024-09-12T03:27:03.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dce8510 0x56521dce5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dcddc50 0x56521dcdfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/3190181344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dce0930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/3190181344 >> 172.21.15.50:0/3190181344 conn(0x56521dcd98c0 msgr2=0x56521dce6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/3190181344 shutdown_connections 2024-09-12T03:27:03.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/3190181344 wait complete. 2024-09-12T03:27:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 -- start start 2024-09-12T03:27:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.386+0000 7fe6ca3c1740 1 Processor -- start 2024-09-12T03:27:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dd03530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dcddc50 0x56521dce25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 0x56521dce2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56521dcfa830 con 0x56521dcdd330 2024-09-12T03:27:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56521dcfa160 con 0x56521dcddc50 2024-09-12T03:27:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56521dcfaf00 con 0x56521dce8510 2024-09-12T03:27:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6ca3c1740 10 client.0 _ll_register_callbacks cb 0x7fe6b4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dd03530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6c5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 0x56521dce2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6c5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dd03530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44510/0 (socket says 172.21.15.50:44510) 2024-09-12T03:27:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6c5000640 1 -- 172.21.15.50:0/486276771 learned_addr learned my addr 172.21.15.50:0/486276771 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6c5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 0x56521dce2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58122/0 (socket says 172.21.15.50:58122) 2024-09-12T03:27:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.387+0000 7fe6bfe00640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dcddc50 0x56521dce25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5a00640 1 -- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dcddc50 msgr2=0x56521dce25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dcddc50 0x56521dce25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5a00640 1 -- 172.21.15.50:0/486276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 msgr2=0x56521dd03530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dd03530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5a00640 1 -- 172.21.15.50:0/486276771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521dd00fe0 con 0x56521dce8510 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5000640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dd03530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6c5a00640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 0x56521dce2ce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x56521dcdd990 tx=0x7fe6b8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d75cbf44b78862 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6a6a00640 1 -- 172.21.15.50:0/486276771 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b802e060 con 0x56521dce8510 2024-09-12T03:27:03.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b40c0030 con 0x56521dce8510 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.388+0000 7fe6a6a00640 1 -- 172.21.15.50:0/486276771 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe6b801e630 con 0x56521dce8510 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6a6a00640 1 -- 172.21.15.50:0/486276771 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6b8035050 con 0x56521dce8510 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 10 client.25129 fetch_fsmap learned FSMap version 25 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56521dd63e20 con 0x56521dce8510 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6a6a00640 1 -- 172.21.15.50:0/486276771 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40c0030 con 0x56521dce8510 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6a6a00640 1 -- 172.21.15.50:0/486276771 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe6b801dab0 con 0x56521dce8510 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 10 client.25129 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 10 client.25129 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:27:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 20 client.25129 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dd689d0 0x56521dd676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 4 client.25129 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.389+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "status"}) -- 0x56521dc4ab30 con 0x56521dd689d0 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.390+0000 7fe6c5000640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dd689d0 0x56521dd676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.390+0000 7fe6c5000640 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dd689d0 0x56521dd676e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad188bcd98d9e2a server_cookie=aa7081bfd6ed7d85 in_seq=0 out_seq=0 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.390+0000 7fe6a6a00640 10 client.25129 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.390+0000 7fe6a6a00640 1 -- 172.21.15.50:0/486276771 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56521dc4ab30 con 0x56521dd689d0 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.390+0000 7fe6a6a00640 10 client.25129 handle_command_reply: tid=0 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.390+0000 7fe6ca3c1740 1 client.25129 shutdown 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 msgr2=0x56521dce2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 0x56521dce2ce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x56521dcdd990 tx=0x7fe6b8004a40 comp rx=0 tx=0).stop 2024-09-12T03:27:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 shutdown_connections 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x56521dd689d0 0x56521dd676e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56521dce8510 0x56521dce2ce0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56521dcddc50 0x56521dce25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 --2- 172.21.15.50:0/486276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56521dcdd330 0x56521dd03530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6a6a00640 0 client.25129 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 >> 172.21.15.50:0/486276771 conn(0x56521dcd98c0 msgr2=0x56521dcf3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 shutdown_connections 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 1 -- 172.21.15.50:0/486276771 wait complete. 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.391+0000 7fe6ca3c1740 20 client.25129 trim_cache size 0 max 16384 2024-09-12T03:27:03.405 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "389f22da-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24536, 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 149.754000674, 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 25, 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 94, 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 163.34449389900001 2024-09-12T03:27:03.406 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:27:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe6a80a2b80 msgr2=0x7fe6a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe6a80a2b80 0x7fe6a80a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8005fd0 tx=0x7fe6b80096a0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 msgr2=0x7fe6c019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 0x7fe6c019a680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4004770 tx=0x7fe6b4003c10 comp rx=0 tx=0).stop 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 shutdown_connections 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7fe6a80a2b80 0x7fe6a80a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c010e3c0 0x7fe6c019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6c0074f40 0x7fe6c019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 --2- 172.21.15.50:0/1165666487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0074620 0x7fe6c019a680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 >> 172.21.15.50:0/1165666487 conn(0x7fe6c00fd8c0 msgr2=0x7fe6c00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 shutdown_connections 2024-09-12T03:27:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:03.392+0000 7fe6c7a00640 1 -- 172.21.15.50:0/1165666487 wait complete. 2024-09-12T03:27:04.083 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:03 smithi050 ceph-mon[30378]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:27:04.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:03 smithi082 ceph-mon[35046]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:27:04.122 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:27:04.132 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:27:04.132 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:27:04.132 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:27:04.132 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-12T03:27:04.133 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:27:04.133 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:27:04.134 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-12T03:27:04.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.288+0000 7f92a9e00640 1 -- 172.21.15.50:0/3419065102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4103d60 msgr2=0x7f92a4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.288+0000 7f92a9e00640 1 --2- 172.21.15.50:0/3419065102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4103d60 0x7f92a4104160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f929402a410 comp rx=0 tx=0).stop 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 -- 172.21.15.50:0/3419065102 shutdown_connections 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 --2- 172.21.15.50:0/3419065102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a4105910 0x7f92a410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 --2- 172.21.15.50:0/3419065102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4104f50 0x7f92a41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 --2- 172.21.15.50:0/3419065102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4103d60 0x7f92a4104160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 -- 172.21.15.50:0/3419065102 >> 172.21.15.50:0/3419065102 conn(0x7f92a40ff4e0 msgr2=0x7f92a4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 -- 172.21.15.50:0/3419065102 shutdown_connections 2024-09-12T03:27:04.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.289+0000 7f92a9e00640 1 -- 172.21.15.50:0/3419065102 wait complete. 2024-09-12T03:27:04.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 Processor -- start 2024-09-12T03:27:04.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 -- start start 2024-09-12T03:27:04.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 0x7f92a419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:04.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a4104f50 0x7f92a419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:04.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 0x7f92a41a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:04.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410fa30 con 0x7f92a4105910 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410f8b0 con 0x7f92a4104f50 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a9e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410fbb0 con 0x7f92a4103d60 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 0x7f92a41a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 0x7f92a419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 0x7f92a41a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44536/0 (socket says 172.21.15.50:44536) 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a3400640 1 -- 172.21.15.50:0/1483579645 learned_addr learned my addr 172.21.15.50:0/1483579645 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 0x7f92a419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58128/0 (socket says 172.21.15.50:58128) 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.290+0000 7f92a2000640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a4104f50 0x7f92a419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a2a00640 1 -- 172.21.15.50:0/1483579645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a4104f50 msgr2=0x7f92a419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a2a00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a4104f50 0x7f92a419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a2a00640 1 -- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 msgr2=0x7f92a41a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a2a00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 0x7f92a41a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a2a00640 1 -- 172.21.15.50:0/1483579645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a41a1c20 con 0x7f92a4103d60 2024-09-12T03:27:04.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a3400640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 0x7f92a41a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a2a00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 0x7f92a419a000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9294009230 tx=0x7f9294032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929403f030 con 0x7f92a4103d60 2024-09-12T03:27:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a41a1eb0 con 0x7f92a4103d60 2024-09-12T03:27:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9294003ce0 con 0x7f92a4103d60 2024-09-12T03:27:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294035070 con 0x7f92a4103d60 2024-09-12T03:27:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.291+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41a22b0 con 0x7f92a4103d60 2024-09-12T03:27:04.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:03 smithi142 ceph-mon[35329]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:27:04.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.292+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a410b080 con 0x7f92a4103d60 2024-09-12T03:27:04.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.295+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9294042030 con 0x7f92a4103d60 2024-09-12T03:27:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.296+0000 7f92a8a00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f92880a2c30 0x7f92880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.296+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f92940efbc0 con 0x7f92a4103d60 2024-09-12T03:27:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.296+0000 7f92a2000640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f92880a2c30 0x7f92880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:04.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.297+0000 7f92a2000640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f92880a2c30 0x7f92880a50f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92a419b510 tx=0x7f9290005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:04.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.298+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92940bc0c0 con 0x7f92a4103d60 2024-09-12T03:27:04.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.531+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f92a4062e90 con 0x7f92a4103d60 2024-09-12T03:27:04.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.818+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f92940b8730 con 0x7f92a4103d60 2024-09-12T03:27:04.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.819+0000 7f92a8a00640 1 -- 172.21.15.50:0/1483579645 <== mon.1 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f92940f1390 con 0x7f92a4103d60 2024-09-12T03:27:04.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f92880a2c30 msgr2=0x7f92880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:04.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f92880a2c30 0x7f92880a50f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92a419b510 tx=0x7f9290005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 msgr2=0x7f92a419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 0x7f92a419a000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9294009230 tx=0x7f9294032ce0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 shutdown_connections 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f92880a2c30 0x7f92880a50f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92a4105910 0x7f92a41a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a4104f50 0x7f92a419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 --2- 172.21.15.50:0/1483579645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92a4103d60 0x7f92a419a000 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 >> 172.21.15.50:0/1483579645 conn(0x7f92a40ff4e0 msgr2=0x7f92a4108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 shutdown_connections 2024-09-12T03:27:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:04.822+0000 7f92a9e00640 1 -- 172.21.15.50:0/1483579645 wait complete. 2024-09-12T03:27:04.843 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T03:27:04.857 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T03:27:04.857 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T03:27:04.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:04.857 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-12T03:27:04.894 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-12T03:27:04.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:04.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:27:04.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:04.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:04 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1483579645' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:04 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: ignoring --setuser ceph since I am not root 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: ignoring --setgroup ceph since I am not root 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:04 smithi082 ceph-mgr[36417]: -- 172.21.15.82:0/3804911106 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db3c0cb0e0 con 0x55db3c0a8800 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:04.963+0000 7f1f22bb6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:27:05.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.021+0000 7f1f22bb6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:27:05.145 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:04 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1483579645' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:27:05.145 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:04 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:27:05.145 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:04 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ignoring --setuser ceph since I am not root 2024-09-12T03:27:05.146 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:04 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ignoring --setgroup ceph since I am not root 2024-09-12T03:27:05.146 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:05.071+0000 7fda41e34040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:27:05.146 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:05.144+0000 7fda41e34040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:27:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:04 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1483579645' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:27:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:04 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:27:05.171 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: ignoring --setuser ceph since I am not root 2024-09-12T03:27:05.171 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: ignoring --setgroup ceph since I am not root 2024-09-12T03:27:05.171 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:04 smithi142 ceph-mgr[37139]: -- 172.21.15.142:0/2158926737 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620325990e0 con 0x562032577400 2024-09-12T03:27:05.171 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:04.964+0000 7fc068a82040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:27:05.171 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.021+0000 7fc068a82040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:27:05.171 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.170+0000 7fc068a82040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:27:05.383 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-12T03:27:05.383 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:27:05.383 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:05.383 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:27:05.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:05.337+0000 7fda41e34040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:27:05.521 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.169+0000 7f1f22bb6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:27:05.521 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.234+0000 7f1f22bb6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:27:05.521 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.382+0000 7f1f22bb6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:27:05.525 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.236+0000 7fc068a82040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:27:05.525 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.384+0000 7fc068a82040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:27:05.525 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.525+0000 7fc068a82040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:27:05.774 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:05.789 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:05.424+0000 7fda41e34040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:27:05.789 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:05.607+0000 7fda41e34040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:27:05.823 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.520+0000 7f1f22bb6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:27:06.067 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:27:06.067 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-mon[30378]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T03:27:06.068 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:05.788+0000 7fda41e34040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:27:06.075 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:27:06.075 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-mon[35046]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T03:27:06.076 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.830+0000 7f1f22bb6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:27:06.076 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:05.891+0000 7f1f22bb6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:27:06.085 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.838+0000 7fc068a82040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:27:06.086 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:05.899+0000 7fc068a82040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:27:06.086 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:27:06.086 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:05 smithi142 ceph-mon[35329]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T03:27:06.331 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:06.178+0000 7fda41e34040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:27:06.331 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:06.253+0000 7fda41e34040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 -- 172.21.15.50:0/2984900884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 msgr2=0x7f739c110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 --2- 172.21.15.50:0/2984900884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c110800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7394009ca0 tx=0x7f739402a950 comp rx=0 tx=0).stop 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 -- 172.21.15.50:0/2984900884 shutdown_connections 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 --2- 172.21.15.50:0/2984900884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c110800 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 --2- 172.21.15.50:0/2984900884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074f40 0x7f739c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 --2- 172.21.15.50:0/2984900884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074620 0x7f739c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.339+0000 7f73a2e00640 1 -- 172.21.15.50:0/2984900884 >> 172.21.15.50:0/2984900884 conn(0x7f739c0fd8b0 msgr2=0x7f739c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.340+0000 7f73a2e00640 1 -- 172.21.15.50:0/2984900884 shutdown_connections 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.340+0000 7f73a2e00640 1 -- 172.21.15.50:0/2984900884 wait complete. 2024-09-12T03:27:06.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.340+0000 7f73a2e00640 1 Processor -- start 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 -- start start 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074620 0x7f739c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 0x7f739c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f739c114140 con 0x7f739c074f40 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f739c113fc0 con 0x7f739c10e3b0 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a2e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1142c0 con 0x7f739c074620 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f739be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074620 0x7f739c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f739b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 0x7f739c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f739be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074620 0x7f739c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58146/0 (socket says 172.21.15.50:58146) 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f739be00640 1 -- 172.21.15.50:0/1989242493 learned_addr learned my addr 172.21.15.50:0/1989242493 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f739b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 0x7f739c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44576/0 (socket says 172.21.15.50:44576) 2024-09-12T03:27:06.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.341+0000 7f73a0e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f73a0e00640 1 -- 172.21.15.50:0/1989242493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074620 msgr2=0x7f739c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f73a0e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074620 0x7f739c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f73a0e00640 1 -- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 msgr2=0x7f739c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f73a0e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 0x7f739c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f73a0e00640 1 -- 172.21.15.50:0/1989242493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c19f5f0 con 0x7f739c10e3b0 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f739b400640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 0x7f739c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:06.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.342+0000 7f73a0e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c19ef10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7394005cc0 tx=0x7f739402f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:06.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.343+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394034040 con 0x7f739c10e3b0 2024-09-12T03:27:06.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.343+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7394030920 con 0x7f739c10e3b0 2024-09-12T03:27:06.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.343+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940045c0 con 0x7f739c10e3b0 2024-09-12T03:27:06.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.343+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739c19f880 con 0x7f739c10e3b0 2024-09-12T03:27:06.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.343+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739c19fd10 con 0x7f739c10e3b0 2024-09-12T03:27:06.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.344+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739c074a00 con 0x7f739c10e3b0 2024-09-12T03:27:06.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.346+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f7394037020 con 0x7f739c10e3b0 2024-09-12T03:27:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.347+0000 7f7398c00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 0x7f73800a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.347+0000 7f739be00640 1 -- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 msgr2=0x7f73800a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/63385930 2024-09-12T03:27:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.347+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f73940f0160 con 0x7f739c10e3b0 2024-09-12T03:27:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.347+0000 7f739be00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 0x7f73800a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:27:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.351+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73940bccb0 con 0x7f739c10e3b0 2024-09-12T03:27:06.353 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:06.075+0000 7f1f22bb6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:27:06.543 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:06.085+0000 7fc068a82040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:27:06.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.548+0000 7f739be00640 1 -- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 msgr2=0x7f73800a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/63385930 2024-09-12T03:27:06.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.548+0000 7f739be00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 0x7f73800a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:27:06.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.574+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f739c02d7c0 con 0x7f739c10e3b0 2024-09-12T03:27:06.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.574+0000 7f7398c00640 1 -- 172.21.15.50:0/1989242493 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f73940098b0 con 0x7f739c10e3b0 2024-09-12T03:27:06.575 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T03:27:06.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.576+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 msgr2=0x7f73800a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.576+0000 7f73a2e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 0x7f73800a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.576+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 msgr2=0x7f739c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.576+0000 7f73a2e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c19ef10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7394005cc0 tx=0x7f739402f3a0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 shutdown_connections 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f73800a2c50 0x7f73800a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c10e3b0 0x7f739c19ef10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f739c074f40 0x7f739c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 --2- 172.21.15.50:0/1989242493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f739c074620 0x7f739c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 >> 172.21.15.50:0/1989242493 conn(0x7f739c0fd8b0 msgr2=0x7f739c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 shutdown_connections 2024-09-12T03:27:06.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:06.577+0000 7f73a2e00640 1 -- 172.21.15.50:0/1989242493 wait complete. 2024-09-12T03:27:06.591 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:06.479+0000 7fda41e34040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:27:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:06 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1989242493' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:27:07.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:06 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1989242493' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:27:07.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:06.955+0000 7f1f22bb6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:27:07.104 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:07.023+0000 7f1f22bb6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:27:07.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:06 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1989242493' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:27:07.294 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:06.969+0000 7fc068a82040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:27:07.294 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:07.037+0000 7fc068a82040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:27:07.328 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:27:07.328 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:27:07.328 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-09-12T03:27:07.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:27:07.329 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:07.329 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:27:07.329 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:27:07.329 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:27:07.329 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:27:07.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:07.329 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-09-12T03:27:07.405 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:27:07.406 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42080sec preferred_lft 42080sec 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:07.407 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-09-12T03:27:07.408 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:27:07.408 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:07.408 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:27:07.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:27:07.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-09-12T03:27:07.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-09-12T03:27:07.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:27:07.409 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:27:07.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:07.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:07.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:07.637 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:27:07.693 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-12T03:27:07.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:07.697 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-09-12T03:27:07.735 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-09-12T03:27:07.735 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:27:07.735 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:27:07.735 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:27:07.735 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:27:07.736 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:07.736 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:27:07.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:27:07.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:27:07.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:27:07.736 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:27:07.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:07.873 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:07.662+0000 7fda41e34040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:27:07.873 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:07.749+0000 7fda41e34040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:27:07.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:07.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:07.942 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-12T03:27:07.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:07.979 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-09-12T03:27:07.985 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:07.727+0000 7f1f22bb6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:27:07.985 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:07.842+0000 7f1f22bb6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:27:08.004 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:07.744+0000 7fc068a82040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:27:08.004 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:07.859+0000 7fc068a82040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:27:08.056 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:08.056 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:27:08.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:27:08.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:27:08.056 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:27:08.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:08.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:08.240 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:27:08.240 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:27:08.241 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:27:08.260 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:07.985+0000 7f1f22bb6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:27:08.261 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.046+0000 7f1f22bb6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:27:08.261 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.193+0000 7f1f22bb6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:27:08.261 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.260+0000 7f1f22bb6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:27:08.284 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.004+0000 7fc068a82040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:27:08.284 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.066+0000 7fc068a82040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:27:08.284 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.216+0000 7fc068a82040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:27:08.284 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.283+0000 7fc068a82040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:27:08.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:08.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:08.478 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:08.478 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-09-12T03:27:08.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-09-12T03:27:08.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:27:08.478 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-09-12T03:27:08.543 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.320+0000 7f1f22bb6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:27:08.543 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.478+0000 7f1f22bb6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:27:08.543 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.343+0000 7fc068a82040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:27:08.543 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.501+0000 7fc068a82040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:27:08.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:08.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:08.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:27:08.654 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:08.697 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:27:08.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:27:08.697 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:08.800 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T03:27:08.800 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T03:27:08.800 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/d81a0432-0cba-4da2-80bc-4beb3a972d69 2024-09-12T03:27:08.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:08.800 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/d81a0432-0cba-4da2-80bc-4beb3a972d69 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-12T03:27:08.811 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.543+0000 7f1f22bb6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:27:08.811 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.608+0000 7f1f22bb6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:27:08.811 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.690+0000 7f1f22bb6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:27:08.811 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.749+0000 7f1f22bb6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:27:08.811 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:08.807+0000 7f1f22bb6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:27:08.839 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.566+0000 7fc068a82040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:27:08.839 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.633+0000 7fc068a82040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:27:08.839 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.715+0000 7fc068a82040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:27:08.839 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.776+0000 7fc068a82040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:27:08.839 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:08.835+0000 7fc068a82040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:27:08.968 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-12T03:27:08.968 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-12T03:27:09.074 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:08.770+0000 7fda41e34040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:27:09.074 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:08.918+0000 7fda41e34040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:27:09.364 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.099+0000 7fda41e34040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:27:09.365 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.177+0000 7fda41e34040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:27:09.365 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.363+0000 7fda41e34040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:27:09.621 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.450+0000 7fda41e34040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:27:09.622 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.526+0000 7fda41e34040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:27:09.628 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-12T03:27:09.628 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-12T03:27:09.628 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.82:6789,172.21.15.142:6789" 2024-09-12T03:27:09.628 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with new device syntax: 0@389f22da-70b5-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/d81a0432-0cba-4da2-80bc-4beb3a972d69 2024-09-12T03:27:09.628 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.50:6789/172.21.15.82:6789/172.21.15.142:6789" will pass to kernel 2024-09-12T03:27:09.632 INFO:teuthology.orchestra.run.smithi050.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:27:09.632 INFO:teuthology.orchestra.run.smithi050.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:27:09.632 INFO:teuthology.orchestra.run.smithi050.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:27:09.632 INFO:teuthology.orchestra.run.smithi050.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:27:09.632 INFO:teuthology.orchestra.run.smithi050.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:27:09.636 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:27:09.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:09.636 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:09.704 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:27:09.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:27:09.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:09.798 INFO:teuthology.orchestra.run.smithi050.stdout:client25159 2024-09-12T03:27:09.799 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:27:09.799 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:27:09.837 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T03:27:09.596+0000 7f1f22bb6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: Standby manager daemon y restarted 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: Standby manager daemon y started 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: Standby manager daemon z restarted 2024-09-12T03:27:09.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: Standby manager daemon z started 2024-09-12T03:27:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:27:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:27:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:27:09.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:09 smithi082 ceph-mon[35046]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.727+0000 7fda41e34040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.812+0000 7fda41e34040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: Standby manager daemon y restarted 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: Standby manager daemon y started 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:27:09.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:27:09.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: Standby manager daemon z restarted 2024-09-12T03:27:09.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: Standby manager daemon z started 2024-09-12T03:27:09.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:27:09.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:27:09.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:27:09.900 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-mon[30378]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:27:09.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:27:09.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:09.970 INFO:teuthology.orchestra.run.smithi050.stdout:client25159 2024-09-12T03:27:09.970 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:27:10.043 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z[37135]: 2024-09-12T03:27:09.628+0000 7fc068a82040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: Standby manager daemon y restarted 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: Standby manager daemon y started 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.82:0/322485348' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: Standby manager daemon z restarted 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: Standby manager daemon z started 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:27:10.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:09 smithi142 ceph-mon[35329]: from='mgr.? 172.21.15.142:0/2518407400' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:27:10.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:27:10.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:10.103 INFO:teuthology.orchestra.run.smithi050.stdout:client25159 2024-09-12T03:27:10.104 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/389f22da-70b5-11ef-bcea-c7b262605968.client25159/status') 2024-09-12T03:27:10.155 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:09.898+0000 7fda41e34040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:27:10.156 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:10.003+0000 7fda41e34040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:27:10.156 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:10.079+0000 7fda41e34040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:27:10.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:10.205 INFO:teuthology.orchestra.run.smithi050.stdout:instance: client.25159 (3)172.21.15.50:0/2625176468 2024-09-12T03:27:10.205 INFO:teuthology.orchestra.run.smithi050.stdout:blocklisted: false 2024-09-12T03:27:10.206 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records in 2024-09-12T03:27:10.206 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records out 2024-09-12T03:27:10.206 INFO:teuthology.orchestra.run.smithi050.stderr:71 bytes copied, 7.1168e-05 s, 998 kB/s 2024-09-12T03:27:10.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:10.217 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:27:10.227 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:27:10.227 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:27:10.227 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-12T03:27:10.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 -- 172.21.15.50:0/4047594862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500104f50 msgr2=0x7f45001053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 --2- 172.21.15.50:0/4047594862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500104f50 0x7f45001053d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009860 tx=0x7f44ec02a340 comp rx=0 tx=0).stop 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 -- 172.21.15.50:0/4047594862 shutdown_connections 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 --2- 172.21.15.50:0/4047594862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4500105910 0x7f450010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 --2- 172.21.15.50:0/4047594862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500104f50 0x7f45001053d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 --2- 172.21.15.50:0/4047594862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.385+0000 7f4508600640 1 -- 172.21.15.50:0/4047594862 >> 172.21.15.50:0/4047594862 conn(0x7f45000ff4e0 msgr2=0x7f4500101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.386+0000 7f4508600640 1 -- 172.21.15.50:0/4047594862 shutdown_connections 2024-09-12T03:27:10.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.386+0000 7f4508600640 1 -- 172.21.15.50:0/4047594862 wait complete. 2024-09-12T03:27:10.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.386+0000 7f4508600640 1 Processor -- start 2024-09-12T03:27:10.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 -- start start 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4500104f50 0x7f450019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500105910 0x7f45001a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f450010f9e0 con 0x7f4500103d60 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f450010f860 con 0x7f4500105910 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4508600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f450010fb60 con 0x7f4500104f50 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4505c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4505c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42586/0 (socket says 172.21.15.50:42586) 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4505c00640 1 -- 172.21.15.50:0/3804543182 learned_addr learned my addr 172.21.15.50:0/3804543182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4505200640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4500104f50 0x7f450019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.387+0000 7f4506600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500105910 0x7f45001a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4505c00640 1 -- 172.21.15.50:0/3804543182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4500104f50 msgr2=0x7f450019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4505c00640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4500104f50 0x7f450019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4505c00640 1 -- 172.21.15.50:0/3804543182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500105910 msgr2=0x7f45001a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4505c00640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500105910 0x7f45001a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4505c00640 1 -- 172.21.15.50:0/3804543182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45001a1b90 con 0x7f4500103d60 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4505c00640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500199f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9a0 tx=0x7f44f400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f400cd80 con 0x7f4500103d60 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f4010910 con 0x7f4500103d60 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45001a1e80 con 0x7f4500103d60 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.388+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4018a30 con 0x7f4500103d60 2024-09-12T03:27:10.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.389+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a22e0 con 0x7f4500103d60 2024-09-12T03:27:10.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.390+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450010b080 con 0x7f4500103d60 2024-09-12T03:27:10.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.392+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f44f4010ac0 con 0x7f4500103d60 2024-09-12T03:27:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.393+0000 7f44fe000640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 0x7f44e80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.393+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f44f40cf830 con 0x7f4500103d60 2024-09-12T03:27:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.393+0000 7f4505200640 1 -- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 msgr2=0x7f44e80a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/63385930 2024-09-12T03:27:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.393+0000 7f4505200640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 0x7f44e80a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:27:10.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.397+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44f40541b0 con 0x7f4500103d60 2024-09-12T03:27:10.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:10.155+0000 7fda41e34040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:27:10.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.594+0000 7f4505200640 1 -- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 msgr2=0x7f44e80a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/63385930 2024-09-12T03:27:10.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.594+0000 7f4505200640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 0x7f44e80a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:27:10.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.624+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f450002d7c0 con 0x7f4500103d60 2024-09-12T03:27:10.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.668+0000 7f44fe000640 1 -- 172.21.15.50:0/3804543182 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f44f409c380 con 0x7f4500103d60 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 msgr2=0x7f44e80a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 0x7f44e80a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 msgr2=0x7f4500199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500199f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9a0 tx=0x7f44f400ee60 comp rx=0 tx=0).stop 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 shutdown_connections 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f44e80a2c80 0x7f44e80a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4500105910 0x7f45001a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4500104f50 0x7f450019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 --2- 172.21.15.50:0/3804543182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4500103d60 0x7f4500199f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 >> 172.21.15.50:0/3804543182 conn(0x7f45000ff4e0 msgr2=0x7f4500108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:10.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 shutdown_connections 2024-09-12T03:27:10.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.672+0000 7f4508600640 1 -- 172.21.15.50:0/3804543182 wait complete. 2024-09-12T03:27:10.689 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T03:27:10.701 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:27:10.737 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:27:10.737 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:27:10.738 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 -- 172.21.15.50:0/3640584440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74104f50 msgr2=0x7f2b741053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 --2- 172.21.15.50:0/3640584440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74104f50 0x7f2b741053d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 -- 172.21.15.50:0/3640584440 shutdown_connections 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 --2- 172.21.15.50:0/3640584440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74105910 0x7f2b7410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 --2- 172.21.15.50:0/3640584440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74104f50 0x7f2b741053d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 --2- 172.21.15.50:0/3640584440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b74103d60 0x7f2b74104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 -- 172.21.15.50:0/3640584440 >> 172.21.15.50:0/3640584440 conn(0x7f2b740ff4e0 msgr2=0x7f2b74101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:10.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 -- 172.21.15.50:0/3640584440 shutdown_connections 2024-09-12T03:27:10.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.895+0000 7f2b78e00640 1 -- 172.21.15.50:0/3640584440 wait complete. 2024-09-12T03:27:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 Processor -- start 2024-09-12T03:27:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 -- start start 2024-09-12T03:27:10.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b74103d60 0x7f2b74199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74104f50 0x7f2b7419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 0x7f2b741a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7410f9c0 con 0x7f2b74105910 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7410f840 con 0x7f2b74103d60 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.896+0000 7f2b78e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7410fb40 con 0x7f2b74104f50 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 0x7f2b741a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 0x7f2b741a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42602/0 (socket says 172.21.15.50:42602) 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b71400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74104f50 0x7f2b7419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 -- 172.21.15.50:0/1020457079 learned_addr learned my addr 172.21.15.50:0/1020457079 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b71e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b74103d60 0x7f2b74199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 -- 172.21.15.50:0/1020457079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74104f50 msgr2=0x7f2b7419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74104f50 0x7f2b7419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 -- 172.21.15.50:0/1020457079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b74103d60 msgr2=0x7f2b74199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b74103d60 0x7f2b74199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 -- 172.21.15.50:0/1020457079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b741a1ba0 con 0x7f2b74105910 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.897+0000 7f2b72800640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 0x7f2b741a1530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00eaa0 tx=0x7f2b6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.898+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00cdb0 con 0x7f2b74105910 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.898+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c005020 con 0x7f2b74105910 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.898+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b741a1e90 con 0x7f2b74105910 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.898+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c0116b0 con 0x7f2b74105910 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.898+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b741a2350 con 0x7f2b74105910 2024-09-12T03:27:10.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.899+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7410b080 con 0x7f2b74105910 2024-09-12T03:27:10.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.903+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f2b6c005560 con 0x7f2b74105910 2024-09-12T03:27:10.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.904+0000 7f2b6aa00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2b580a2d00 0x7f2b580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.904+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f2b6c0cf5f0 con 0x7f2b74105910 2024-09-12T03:27:10.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.904+0000 7f2b71e00640 1 -- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2b580a2d00 msgr2=0x7f2b580a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/63385930 2024-09-12T03:27:10.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.904+0000 7f2b71e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2b580a2d00 0x7f2b580a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:27:10.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:10.908+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b6c016030 con 0x7f2b74105910 2024-09-12T03:27:11.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:10 smithi142 ceph-mon[35329]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-12T03:27:11.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:10 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3804543182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:27:11.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.094+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f2b74062f00 con 0x7f2b74105910 2024-09-12T03:27:11.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.099+0000 7f2b6aa00640 1 -- 172.21.15.50:0/1020457079 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f2b6c09baf0 con 0x7f2b74105910 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2b580a2d00 msgr2=0x7f2b580a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2b580a2d00 0x7f2b580a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 msgr2=0x7f2b741a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 0x7f2b741a1530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00eaa0 tx=0x7f2b6c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 shutdown_connections 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:6800/63385930,v1:172.21.15.50:6801/63385930] conn(0x7f2b580a2d00 0x7f2b580a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b74105910 0x7f2b741a1530 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74104f50 0x7f2b7419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 --2- 172.21.15.50:0/1020457079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b74103d60 0x7f2b74199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 >> 172.21.15.50:0/1020457079 conn(0x7f2b740ff4e0 msgr2=0x7f2b74108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:11.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 shutdown_connections 2024-09-12T03:27:11.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.101+0000 7f2b78e00640 1 -- 172.21.15.50:0/1020457079 wait complete. 2024-09-12T03:27:11.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:10 smithi082 ceph-mon[35046]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-12T03:27:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:10 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3804543182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:27:11.116 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:10 smithi050 ceph-mon[30378]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-12T03:27:11.116 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:10 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3804543182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:27:11.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-12T03:27:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 -- 172.21.15.50:0/3448624082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 msgr2=0x7f34ac1025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 --2- 172.21.15.50:0/3448624082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac1025c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009900 tx=0x7f34a002a420 comp rx=0 tx=0).stop 2024-09-12T03:27:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 -- 172.21.15.50:0/3448624082 shutdown_connections 2024-09-12T03:27:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 --2- 172.21.15.50:0/3448624082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34ac1054b0 0x7f34ac105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 --2- 172.21.15.50:0/3448624082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 --2- 172.21.15.50:0/3448624082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac1025c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.305+0000 7f34b2000640 1 -- 172.21.15.50:0/3448624082 >> 172.21.15.50:0/3448624082 conn(0x7f34ac0fbfa0 msgr2=0x7f34ac0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:11.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.306+0000 7f34b2000640 1 -- 172.21.15.50:0/3448624082 shutdown_connections 2024-09-12T03:27:11.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.306+0000 7f34b2000640 1 -- 172.21.15.50:0/3448624082 wait complete. 2024-09-12T03:27:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.308+0000 7f34b2000640 1 Processor -- start 2024-09-12T03:27:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.308+0000 7f34b2000640 1 -- start start 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.308+0000 7f34b2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.308+0000 7f34b2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.308+0000 7f34b2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34ac1054b0 0x7f34ac1a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.309+0000 7f34b0c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.309+0000 7f34b0c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:48882/0 (socket says 172.21.15.50:48882) 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.309+0000 7f34abe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:11.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.309+0000 7f34abe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41734/0 (socket says 172.21.15.50:41734) 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac113c30 con 0x7f34ac1054b0 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac113ab0 con 0x7f34ac1001b0 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac113db0 con 0x7f34ac102b00 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b0c00640 1 -- 172.21.15.50:0/687030350 learned_addr learned my addr 172.21.15.50:0/687030350 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b0c00640 1 -- 172.21.15.50:0/687030350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 msgr2=0x7f34ac19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b0c00640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b0c00640 1 -- 172.21.15.50:0/687030350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34ac1054b0 msgr2=0x7f34ac1a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b0c00640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34ac1054b0 0x7f34ac1a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34b0c00640 1 -- 172.21.15.50:0/687030350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac1a5f40 con 0x7f34ac1001b0 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.310+0000 7f34abe00640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.311+0000 7f34b0c00640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac19e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005880 tx=0x7f34a00049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.312+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0040070 con 0x7f34ac1001b0 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.312+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a0004dd0 con 0x7f34ac1001b0 2024-09-12T03:27:11.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.312+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0035020 con 0x7f34ac1001b0 2024-09-12T03:27:11.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.313+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac1a61d0 con 0x7f34ac1001b0 2024-09-12T03:27:11.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.313+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1a6670 con 0x7f34ac1001b0 2024-09-12T03:27:11.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.314+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac0fda90 con 0x7f34ac1001b0 2024-09-12T03:27:11.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.318+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f34a0043070 con 0x7f34ac1001b0 2024-09-12T03:27:11.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.321+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f34a00efca0 con 0x7f34ac1001b0 2024-09-12T03:27:11.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.323+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34a00bcaf0 con 0x7f34ac1001b0 2024-09-12T03:27:11.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.141+0000 7fda41e34040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:27:11.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.626+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f34ac062e90 con 0x7f34ac1001b0 2024-09-12T03:27:11.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.633+0000 7f34a9600640 1 -- 172.21.15.50:0/687030350 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f34a00bc3f0 con 0x7f34ac1001b0 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 msgr2=0x7f34ac19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac19e320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005880 tx=0x7f34a00049d0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 shutdown_connections 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34ac1054b0 0x7f34ac1a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ac102b00 0x7f34ac19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 --2- 172.21.15.50:0/687030350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac1001b0 0x7f34ac19e320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.636+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 >> 172.21.15.50:0/687030350 conn(0x7f34ac0fbfa0 msgr2=0x7f34ac1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:11.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.637+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 shutdown_connections 2024-09-12T03:27:11.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.637+0000 7f34b2000640 1 -- 172.21.15.50:0/687030350 wait complete. 2024-09-12T03:27:11.661 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-12T03:27:11.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 -- 172.21.15.50:0/2467952700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 msgr2=0x7f61881031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 --2- 172.21.15.50:0/2467952700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f61881031b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6174009900 tx=0x7f617402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 -- 172.21.15.50:0/2467952700 shutdown_connections 2024-09-12T03:27:11.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 --2- 172.21.15.50:0/2467952700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61881036f0 0x7f6188109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 --2- 172.21.15.50:0/2467952700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f61881031b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 --2- 172.21.15.50:0/2467952700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6188101b40 0x7f6188101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 -- 172.21.15.50:0/2467952700 >> 172.21.15.50:0/2467952700 conn(0x7f61880fd2e0 msgr2=0x7f61880ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:11.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 -- 172.21.15.50:0/2467952700 shutdown_connections 2024-09-12T03:27:11.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.835+0000 7f618f600640 1 -- 172.21.15.50:0/2467952700 wait complete. 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 Processor -- start 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 -- start start 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6188101b40 0x7f6188197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f6188198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61881036f0 0x7f618819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f618810d810 con 0x7f61881036f0 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f618810d690 con 0x7f6188102d30 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.836+0000 7f618f600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f618810d990 con 0x7f6188101b40 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f6188198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f618d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61881036f0 0x7f618819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f6188198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:48958/0 (socket says 172.21.15.50:48958) 2024-09-12T03:27:11.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 -- 172.21.15.50:0/1631239 learned_addr learned my addr 172.21.15.50:0/1631239 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f618cc00640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6188101b40 0x7f6188197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 -- 172.21.15.50:0/1631239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6188101b40 msgr2=0x7f6188197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6188101b40 0x7f6188197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 -- 172.21.15.50:0/1631239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61881036f0 msgr2=0x7f618819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61881036f0 0x7f618819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 -- 172.21.15.50:0/1631239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618819fa20 con 0x7f6188102d30 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.837+0000 7f6187e00640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f6188198340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6174031440 tx=0x7f6174032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.838+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617403f030 con 0x7f6188102d30 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.838+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6174003ce0 con 0x7f6188102d30 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.838+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174035070 con 0x7f6188102d30 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.838+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618819fcb0 con 0x7f6188102d30 2024-09-12T03:27:11.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.838+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61881a0150 con 0x7f6188102d30 2024-09-12T03:27:11.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.839+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6188108eb0 con 0x7f6188102d30 2024-09-12T03:27:11.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.842+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f6174042030 con 0x7f6188102d30 2024-09-12T03:27:11.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.844+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f61740ef660 con 0x7f6188102d30 2024-09-12T03:27:11.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:11.845+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61740bbe60 con 0x7f6188102d30 2024-09-12T03:27:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3804543182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:27:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1020457079' entity='client.admin' 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: Active manager daemon x restarted 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: Activating manager daemon x 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: mgrmap e21: x(active, starting, since 0.0123394s), standbys: z, y 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:27:11.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:27:11.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:27:11.857 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: Manager daemon x is now available 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:27:11.858 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:11 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' 2024-09-12T03:27:11.866 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3804543182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:27:11.866 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:11.866 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1020457079' entity='client.admin' 2024-09-12T03:27:11.866 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: Active manager daemon x restarted 2024-09-12T03:27:11.866 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: Activating manager daemon x 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: mgrmap e21: x(active, starting, since 0.0123394s), standbys: z, y 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:27:11.867 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:27:11.868 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: Manager daemon x is now available 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:11.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:27:11.870 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:27:11.870 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:27:11.870 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:27:11.870 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:11 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' 2024-09-12T03:27:11.914 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99ee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99ee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99ee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99ee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99ee00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99a800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99a800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99a800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99a800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.915 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:11.541+0000 7fd99a800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3804543182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1020457079' entity='client.admin' 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: Active manager daemon x restarted 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: Activating manager daemon x 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: mgrmap e21: x(active, starting, since 0.0123394s), standbys: z, y 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:27:11.916 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:27:11.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:27:11.918 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: Manager daemon x is now available 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:27:11.919 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:11 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' 2024-09-12T03:27:12.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.046+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f618802d7c0 con 0x7f6188102d30 2024-09-12T03:27:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.051+0000 7f6185600640 1 -- 172.21.15.50:0/1631239 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f61740c0160 con 0x7f6188102d30 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 msgr2=0x7f6188198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f6188198340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6174031440 tx=0x7f6174032ce0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 shutdown_connections 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61881036f0 0x7f618819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6188102d30 0x7f6188198340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 --2- 172.21.15.50:0/1631239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6188101b40 0x7f6188197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 >> 172.21.15.50:0/1631239 conn(0x7f61880fd2e0 msgr2=0x7f6188106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 shutdown_connections 2024-09-12T03:27:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.054+0000 7f618f600640 1 -- 172.21.15.50:0/1631239 wait complete. 2024-09-12T03:27:12.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 -- 172.21.15.50:0/81083525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270103d60 msgr2=0x7f4270104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 --2- 172.21.15.50:0/81083525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270103d60 0x7f4270104160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4260009900 tx=0x7f426002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 -- 172.21.15.50:0/81083525 shutdown_connections 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 --2- 172.21.15.50:0/81083525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270105910 0x7f427010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 --2- 172.21.15.50:0/81083525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4270104f50 0x7f42701053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 --2- 172.21.15.50:0/81083525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270103d60 0x7f4270104160 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 -- 172.21.15.50:0/81083525 >> 172.21.15.50:0/81083525 conn(0x7f42700ff4e0 msgr2=0x7f4270101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 -- 172.21.15.50:0/81083525 shutdown_connections 2024-09-12T03:27:12.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.258+0000 7f4274c00640 1 -- 172.21.15.50:0/81083525 wait complete. 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 Processor -- start 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 -- start start 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4270103d60 0x7f4270199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270104f50 0x7f427019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 0x7f42701a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f427010fa10 con 0x7f4270105910 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f427010f890 con 0x7f4270103d60 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.259+0000 7f4274c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f427010fb90 con 0x7f4270104f50 2024-09-12T03:27:12.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270104f50 0x7f427019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 0x7f42701a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4270103d60 0x7f4270199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 0x7f42701a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42810/0 (socket says 172.21.15.50:42810) 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426e800640 1 -- 172.21.15.50:0/497971678 learned_addr learned my addr 172.21.15.50:0/497971678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426d400640 1 -- 172.21.15.50:0/497971678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4270103d60 msgr2=0x7f4270199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426d400640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4270103d60 0x7f4270199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426d400640 1 -- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 msgr2=0x7f42701a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426d400640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 0x7f42701a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426d400640 1 -- 172.21.15.50:0/497971678 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42701a1bf0 con 0x7f4270104f50 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.260+0000 7f426e800640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 0x7f42701a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:12.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.261+0000 7f426d400640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270104f50 0x7f427019a510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f425c00e9f0 tx=0x7f425c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:12.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.261+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c00ccc0 con 0x7f4270104f50 2024-09-12T03:27:12.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.261+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f425c010910 con 0x7f4270104f50 2024-09-12T03:27:12.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.261+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c018960 con 0x7f4270104f50 2024-09-12T03:27:12.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.261+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42701a1ee0 con 0x7f4270104f50 2024-09-12T03:27:12.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.261+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a2350 con 0x7f4270104f50 2024-09-12T03:27:12.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.262+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427010b080 con 0x7f4270104f50 2024-09-12T03:27:12.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.265+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f425c010ac0 con 0x7f4270104f50 2024-09-12T03:27:12.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.267+0000 7f4266a00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f42540a2ac0 0x7f42540a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.267+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f425c0cf3a0 con 0x7f4270104f50 2024-09-12T03:27:12.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.267+0000 7f426de00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f42540a2ac0 0x7f42540a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.268+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f425c09bab0 con 0x7f4270104f50 2024-09-12T03:27:12.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.268+0000 7f426de00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f42540a2ac0 0x7f42540a4f80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f426002a9e0 tx=0x7f42600315b0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:12.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.484+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 --> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4270062630 con 0x7f42540a2ac0 2024-09-12T03:27:12.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.560+0000 7f4266a00640 1 -- 172.21.15.50:0/497971678 <== mgr.25157 v2:172.21.15.50:6800/682249052 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4270062630 con 0x7f42540a2ac0 2024-09-12T03:27:12.561 INFO:teuthology.orchestra.run.smithi050.stdout:Schedule set for path / 2024-09-12T03:27:12.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f42540a2ac0 msgr2=0x7f42540a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f42540a2ac0 0x7f42540a4f80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f426002a9e0 tx=0x7f42600315b0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270104f50 msgr2=0x7f427019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270104f50 0x7f427019a510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f425c00e9f0 tx=0x7f425c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 shutdown_connections 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f42540a2ac0 0x7f42540a4f80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4270105910 0x7f42701a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4270104f50 0x7f427019a510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 --2- 172.21.15.50:0/497971678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4270103d60 0x7f4270199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 >> 172.21.15.50:0/497971678 conn(0x7f42700ff4e0 msgr2=0x7f4270108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.562+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 shutdown_connections 2024-09-12T03:27:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.563+0000 7f4274c00640 1 -- 172.21.15.50:0/497971678 wait complete. 2024-09-12T03:27:12.578 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-12T03:27:12.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 -- 172.21.15.50:0/777434557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc076bb0 msgr2=0x7fcedc076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 --2- 172.21.15.50:0/777434557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc076bb0 0x7fcedc076fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009860 tx=0x7fcecc02a340 comp rx=0 tx=0).stop 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 -- 172.21.15.50:0/777434557 shutdown_connections 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 --2- 172.21.15.50:0/777434557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc078790 0x7fcedc079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 --2- 172.21.15.50:0/777434557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcedc077df0 0x7fcedc078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 --2- 172.21.15.50:0/777434557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc076bb0 0x7fcedc076fb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 -- 172.21.15.50:0/777434557 >> 172.21.15.50:0/777434557 conn(0x7fcedc0ff4e0 msgr2=0x7fcedc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 -- 172.21.15.50:0/777434557 shutdown_connections 2024-09-12T03:27:12.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.752+0000 7fcee2a00640 1 -- 172.21.15.50:0/777434557 wait complete. 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.753+0000 7fcee2a00640 1 Processor -- start 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.753+0000 7fcee2a00640 1 -- start start 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 0x7fcedc19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc077df0 0x7fcedc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcedc078790 0x7fcedc1a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee2a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc113df0 con 0x7fcedc078790 2024-09-12T03:27:12.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc113c70 con 0x7fcedc077df0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee2a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc113f70 con 0x7fcedc076bb0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 0x7fcedc19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedb400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc077df0 0x7fcedc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 0x7fcedc19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41822/0 (socket says 172.21.15.50:41822) 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedb400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc077df0 0x7fcedc19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:49010/0 (socket says 172.21.15.50:49010) 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 -- 172.21.15.50:0/3518223899 learned_addr learned my addr 172.21.15.50:0/3518223899 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcee0a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcedc078790 0x7fcedc1a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 -- 172.21.15.50:0/3518223899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc077df0 msgr2=0x7fcedc19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc077df0 0x7fcedc19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 -- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcedc078790 msgr2=0x7fcedc1a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcedc078790 0x7fcedc1a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.754+0000 7fcedbe00640 1 -- 172.21.15.50:0/3518223899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc1a5ff0 con 0x7fcedc076bb0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.755+0000 7fcedbe00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 0x7fcedc19e3d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcecc035040 tx=0x7fcecc033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:12.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.755+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc037040 con 0x7fcedc076bb0 2024-09-12T03:27:12.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.755+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcecc030850 con 0x7fcedc076bb0 2024-09-12T03:27:12.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.755+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004920 con 0x7fcedc076bb0 2024-09-12T03:27:12.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.755+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc1a6280 con 0x7fcedc076bb0 2024-09-12T03:27:12.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.755+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc1a6720 con 0x7fcedc076bb0 2024-09-12T03:27:12.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.756+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc1106d0 con 0x7fcedc076bb0 2024-09-12T03:27:12.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.758+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fcecc004050 con 0x7fcedc076bb0 2024-09-12T03:27:12.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.759+0000 7fced8c00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fcec00a2a40 0x7fcec00a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.760+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcecc0eeff0 con 0x7fcedc076bb0 2024-09-12T03:27:12.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.760+0000 7fcedb400640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fcec00a2a40 0x7fcec00a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.760+0000 7fcedb400640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fcec00a2a40 0x7fcec00a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcedc19f8e0 tx=0x7fcec8009260 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:12.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:12.764+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcecc0bbd90 con 0x7fcedc076bb0 2024-09-12T03:27:13.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.030+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 --> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fcedc062790 con 0x7fcec00a2a40 2024-09-12T03:27:13.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.096+0000 7fced8c00640 1 -- 172.21.15.50:0/3518223899 <== mgr.25157 v2:172.21.15.50:6800/682249052 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fcedc062790 con 0x7fcec00a2a40 2024-09-12T03:27:13.097 INFO:teuthology.orchestra.run.smithi050.stdout:Retention added to path / 2024-09-12T03:27:13.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fcec00a2a40 msgr2=0x7fcec00a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fcec00a2a40 0x7fcec00a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcedc19f8e0 tx=0x7fcec8009260 comp rx=0 tx=0).stop 2024-09-12T03:27:13.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 msgr2=0x7fcedc19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 0x7fcedc19e3d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcecc035040 tx=0x7fcecc033d90 comp rx=0 tx=0).stop 2024-09-12T03:27:13.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 shutdown_connections 2024-09-12T03:27:13.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fcec00a2a40 0x7fcec00a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcedc078790 0x7fcedc1a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc077df0 0x7fcedc19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 --2- 172.21.15.50:0/3518223899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcedc076bb0 0x7fcedc19e3d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 >> 172.21.15.50:0/3518223899 conn(0x7fcedc0ff4e0 msgr2=0x7fcedc100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 shutdown_connections 2024-09-12T03:27:13.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.099+0000 7fcee2a00640 1 -- 172.21.15.50:0/3518223899 wait complete. 2024-09-12T03:27:13.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.281+0000 7fda2da00640 1 -- 172.21.15.50:0/2535583102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28100a40 msgr2=0x7fda28100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 --2- 172.21.15.50:0/2535583102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28100a40 0x7fda28100e40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fda18009900 tx=0x7fda1802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 -- 172.21.15.50:0/2535583102 shutdown_connections 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 --2- 172.21.15.50:0/2535583102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda281025f0 0x7fda28108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 --2- 172.21.15.50:0/2535583102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda28101c30 0x7fda281020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 --2- 172.21.15.50:0/2535583102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28100a40 0x7fda28100e40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 -- 172.21.15.50:0/2535583102 >> 172.21.15.50:0/2535583102 conn(0x7fda280fc1e0 msgr2=0x7fda280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 -- 172.21.15.50:0/2535583102 shutdown_connections 2024-09-12T03:27:13.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.282+0000 7fda2da00640 1 -- 172.21.15.50:0/2535583102 wait complete. 2024-09-12T03:27:13.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.283+0000 7fda2da00640 1 Processor -- start 2024-09-12T03:27:13.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.283+0000 7fda2da00640 1 -- start start 2024-09-12T03:27:13.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda2da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 0x7fda281049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda2da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda28101c30 0x7fda28103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda2da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda281025f0 0x7fda28103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda2da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fda2810c6f0 con 0x7fda281025f0 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda2da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fda2810c570 con 0x7fda28100a40 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda2da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda2810c870 con 0x7fda28101c30 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda26a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 0x7fda281049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda26a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 0x7fda281049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:49054/0 (socket says 172.21.15.50:49054) 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda27400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda281025f0 0x7fda28103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda26a00640 1 -- 172.21.15.50:0/2492191906 learned_addr learned my addr 172.21.15.50:0/2492191906 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.284+0000 7fda26000640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda28101c30 0x7fda28103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda26a00640 1 -- 172.21.15.50:0/2492191906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda28101c30 msgr2=0x7fda28103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda26a00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda28101c30 0x7fda28103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda26a00640 1 -- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda281025f0 msgr2=0x7fda28103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda26a00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda281025f0 0x7fda28103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda26a00640 1 -- 172.21.15.50:0/2492191906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda28103de0 con 0x7fda28100a40 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda27400640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda281025f0 0x7fda28103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda26a00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 0x7fda281049d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda18031440 tx=0x7fda18032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1803f030 con 0x7fda28100a40 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda18003ce0 con 0x7fda28100a40 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.285+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda281a1e90 con 0x7fda28100a40 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.286+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18035070 con 0x7fda28100a40 2024-09-12T03:27:13.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.286+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda281a22f0 con 0x7fda28100a40 2024-09-12T03:27:13.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.287+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda281a2580 con 0x7fda28100a40 2024-09-12T03:27:13.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.289+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fda18042030 con 0x7fda28100a40 2024-09-12T03:27:13.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.290+0000 7fda1f400640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fda0c0a2ac0 0x7fda0c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.290+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fda180ef4d0 con 0x7fda28100a40 2024-09-12T03:27:13.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.290+0000 7fda26000640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fda0c0a2ac0 0x7fda0c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.291+0000 7fda26000640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fda0c0a2ac0 0x7fda0c0a4f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fda28104740 tx=0x7fda14005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:13.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.294+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda180bbbe0 con 0x7fda28100a40 2024-09-12T03:27:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:13 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' 2024-09-12T03:27:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:13 smithi082 ceph-mon[35046]: mgrmap e22: x(active, since 1.02187s), standbys: z, y 2024-09-12T03:27:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:13 smithi082 ceph-mon[35046]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:13 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:13 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:13 smithi082 ceph-mon[35046]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' 2024-09-12T03:27:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-mon[30378]: mgrmap e22: x(active, since 1.02187s), standbys: z, y 2024-09-12T03:27:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-mon[30378]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-mon[30378]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:13.235+0000 7fd982c00640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:27:13] ENGINE Error in HTTPServer.serve 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Traceback (most recent call last): 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._connections.run(self.expiration_interval) 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._run(expiration_interval) 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:27:13.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: s = self.context.wrap_socket( 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: return self.sslsocket_class._create( 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self.do_handshake() 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self._sslobj.do_handshake() 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:27:13.416 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:27:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:27:13.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.507+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 --> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fda280fe390 con 0x7fda0c0a2ac0 2024-09-12T03:27:13.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.515+0000 7fda1f400640 1 -- 172.21.15.50:0/2492191906 <== mgr.25157 v2:172.21.15.50:6800/682249052 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fda280fe390 con 0x7fda0c0a2ac0 2024-09-12T03:27:13.515 INFO:teuthology.orchestra.run.smithi050.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T03:27:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fda0c0a2ac0 msgr2=0x7fda0c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fda0c0a2ac0 0x7fda0c0a4f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fda28104740 tx=0x7fda14005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 msgr2=0x7fda281049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 0x7fda281049d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda18031440 tx=0x7fda18032ce0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 shutdown_connections 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fda0c0a2ac0 0x7fda0c0a4f80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda281025f0 0x7fda28103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda28101c30 0x7fda28103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 --2- 172.21.15.50:0/2492191906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda28100a40 0x7fda281049d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 >> 172.21.15.50:0/2492191906 conn(0x7fda280fc1e0 msgr2=0x7fda280fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 shutdown_connections 2024-09-12T03:27:13.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.517+0000 7fda2da00640 1 -- 172.21.15.50:0/2492191906 wait complete. 2024-09-12T03:27:13.533 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:27:13.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:13 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' 2024-09-12T03:27:13.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:13 smithi142 ceph-mon[35329]: mgrmap e22: x(active, since 1.02187s), standbys: z, y 2024-09-12T03:27:13.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:13 smithi142 ceph-mon[35329]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:13.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:13 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:13.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:13 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:13.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:13 smithi142 ceph-mon[35329]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 -- 172.21.15.50:0/2336175706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0074600 msgr2=0x7f91d0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 --2- 172.21.15.50:0/2336175706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0074600 0x7f91d0074a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009900 tx=0x7f91c002a410 comp rx=0 tx=0).stop 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 -- 172.21.15.50:0/2336175706 shutdown_connections 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 --2- 172.21.15.50:0/2336175706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d0109d10 0x7f91d010c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 --2- 172.21.15.50:0/2336175706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074f40 0x7f91d01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 --2- 172.21.15.50:0/2336175706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0074600 0x7f91d0074a00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 -- 172.21.15.50:0/2336175706 >> 172.21.15.50:0/2336175706 conn(0x7f91d00fd0d0 msgr2=0x7f91d00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 -- 172.21.15.50:0/2336175706 shutdown_connections 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.694+0000 7f91cfe00640 1 -- 172.21.15.50:0/2336175706 wait complete. 2024-09-12T03:27:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 Processor -- start 2024-09-12T03:27:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 -- start start 2024-09-12T03:27:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 0x7f91d0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d0074f40 0x7f91d019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0109d10 0x7f91d01a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91d010f8d0 con 0x7f91d0074f40 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f91d010f750 con 0x7f91d0109d10 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f91d010fa50 con 0x7f91d0074600 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 0x7f91d0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 0x7f91d0199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41878/0 (socket says 172.21.15.50:41878) 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91cea00640 1 -- 172.21.15.50:0/983256981 learned_addr learned my addr 172.21.15.50:0/983256981 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.695+0000 7f91ce000640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d0074f40 0x7f91d019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cf400640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0109d10 0x7f91d01a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cea00640 1 -- 172.21.15.50:0/983256981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0109d10 msgr2=0x7f91d01a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cea00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0109d10 0x7f91d01a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cea00640 1 -- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d0074f40 msgr2=0x7f91d019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cea00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d0074f40 0x7f91d019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cea00640 1 -- 172.21.15.50:0/983256981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d01a1bb0 con 0x7f91d0074600 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91cea00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 0x7f91d0199f90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009230 tx=0x7f91c00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c003f030 con 0x7f91d0074600 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.696+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c0003ce0 con 0x7f91d0074600 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.697+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d01a1e40 con 0x7f91d0074600 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.697+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0035070 con 0x7f91d0074600 2024-09-12T03:27:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.697+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a22a0 con 0x7f91d0074600 2024-09-12T03:27:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.698+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d01085e0 con 0x7f91d0074600 2024-09-12T03:27:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.699+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f91c0042030 con 0x7f91d0074600 2024-09-12T03:27:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.701+0000 7f91c7400640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f91b40a29f0 0x7f91b40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.701+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f91c00ef500 con 0x7f91d0074600 2024-09-12T03:27:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.701+0000 7f91ce000640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f91b40a29f0 0x7f91b40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.701+0000 7f91ce000640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f91b40a29f0 0x7f91b40a4eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91d019b4a0 tx=0x7f91bc005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:13.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.704+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91c00bbc10 con 0x7f91d0074600 2024-09-12T03:27:13.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.919+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 --> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f91d0062630 con 0x7f91b40a29f0 2024-09-12T03:27:13.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.927+0000 7f91c7400640 1 -- 172.21.15.50:0/983256981 <== mgr.25157 v2:172.21.15.50:6800/682249052 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f91d0062630 con 0x7f91b40a29f0 2024-09-12T03:27:13.928 INFO:teuthology.orchestra.run.smithi050.stdout:/ 1m 6m3h 2024-09-12T03:27:13.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f91b40a29f0 msgr2=0x7f91b40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f91b40a29f0 0x7f91b40a4eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91d019b4a0 tx=0x7f91bc005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:13.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 msgr2=0x7f91d0199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 0x7f91d0199f90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009230 tx=0x7f91c00042f0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 shutdown_connections 2024-09-12T03:27:13.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91d0109d10 0x7f91d01a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f91b40a29f0 0x7f91b40a4eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d0074f40 0x7f91d019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 --2- 172.21.15.50:0/983256981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91d0074600 0x7f91d0199f90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 >> 172.21.15.50:0/983256981 conn(0x7f91d00fd0d0 msgr2=0x7f91d01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 shutdown_connections 2024-09-12T03:27:13.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:27:13.930+0000 7f91cfe00640 1 -- 172.21.15.50:0/983256981 wait complete. 2024-09-12T03:27:13.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-12T03:27:14.015 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-12T03:27:14.028 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T03:27:14.063 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:27:14.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T03:27:14.068 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T03:27:14.101 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:27:14.102 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T03:27:14.102 INFO:tasks.workunit:timeout=3h 2024-09-12T03:27:14.102 INFO:tasks.workunit:cleanup=True 2024-09-12T03:27:14.102 DEBUG:teuthology.orchestra.run.smithi050:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:14.142 INFO:teuthology.orchestra.run.smithi050.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:14.142 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T03:27:14.142 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T03:27:14.143 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:27:14.143 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T03:27:14.144 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-12 03:25:33.100205874 +0000 2024-09-12T03:27:14.144 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-12 03:25:33.100205874 +0000 2024-09-12T03:27:14.145 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-12 03:27:09.696605952 +0000 2024-09-12T03:27:14.145 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-12 03:25:33.100205874 +0000 2024-09-12T03:27:14.146 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T03:27:14.146 DEBUG:teuthology.orchestra.run.smithi050:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T03:27:14.254 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:27:14.294 INFO:tasks.workunit.client.0.smithi050.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T03:27:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: [12/Sep/2024:03:27:13] ENGINE Bus STARTING 2024-09-12T03:27:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: [12/Sep/2024:03:27:13] ENGINE Serving on http://172.21.15.50:8765 2024-09-12T03:27:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: [12/Sep/2024:03:27:13] ENGINE Serving on https://172.21.15.50:7150 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: [12/Sep/2024:03:27:13] ENGINE Bus STARTED 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: [12/Sep/2024:03:27:13] ENGINE Error in HTTPServer.serve 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout: Traceback (most recent call last): 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:27:14.605 INFO:journalctl@ceph.mon.b.smithi082.stdout: self._run(expiration_interval) 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: s = self.context.wrap_socket( 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: return self.sslsocket_class._create( 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: self.do_handshake() 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:27:14.606 INFO:journalctl@ceph.mon.b.smithi082.stdout: self._sslobj.do_handshake() 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='client.25241 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.607 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.608 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:14 smithi082 ceph-mon[35046]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-12T03:27:14.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: [12/Sep/2024:03:27:13] ENGINE Bus STARTING 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: [12/Sep/2024:03:27:13] ENGINE Serving on http://172.21.15.50:8765 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: [12/Sep/2024:03:27:13] ENGINE Serving on https://172.21.15.50:7150 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: [12/Sep/2024:03:27:13] ENGINE Bus STARTED 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: [12/Sep/2024:03:27:13] ENGINE Error in HTTPServer.serve 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: Traceback (most recent call last): 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:27:14.665 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._run(expiration_interval) 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: s = self.context.wrap_socket( 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: return self.sslsocket_class._create( 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: self.do_handshake() 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:27:14.666 INFO:journalctl@ceph.mon.a.smithi050.stdout: self._sslobj.do_handshake() 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='client.25241 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.668 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.669 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:14 smithi050 ceph-mon[30378]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: [12/Sep/2024:03:27:13] ENGINE Bus STARTING 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: [12/Sep/2024:03:27:13] ENGINE Serving on http://172.21.15.50:8765 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: [12/Sep/2024:03:27:13] ENGINE Serving on https://172.21.15.50:7150 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: [12/Sep/2024:03:27:13] ENGINE Bus STARTED 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: [12/Sep/2024:03:27:13] ENGINE Error in HTTPServer.serve 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout: Traceback (most recent call last): 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:27:14.794 INFO:journalctl@ceph.mon.c.smithi142.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: self._run(expiration_interval) 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: s = self.context.wrap_socket( 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:27:14.795 INFO:journalctl@ceph.mon.c.smithi142.stdout: return self.sslsocket_class._create( 2024-09-12T03:27:14.796 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:27:14.796 INFO:journalctl@ceph.mon.c.smithi142.stdout: self.do_handshake() 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout: self._sslobj.do_handshake() 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='client.25241 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.797 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.798 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.799 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.799 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:14.799 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:14 smithi142 ceph-mon[35329]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-12T03:27:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:15 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi082 to 176.0M 2024-09-12T03:27:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:15 smithi082 ceph-mon[35046]: Unable to set osd_memory_target on smithi082 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T03:27:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:15 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-12T03:27:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:15 smithi082 ceph-mon[35046]: Unable to set osd_memory_target on smithi142 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:27:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:15 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:15.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:15 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:15 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi082 to 176.0M 2024-09-12T03:27:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:15 smithi050 ceph-mon[30378]: Unable to set osd_memory_target on smithi082 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T03:27:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:15 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-12T03:27:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:15 smithi050 ceph-mon[30378]: Unable to set osd_memory_target on smithi142 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:27:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:15 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:15.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:15 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:15 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi082 to 176.0M 2024-09-12T03:27:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:15 smithi142 ceph-mon[35329]: Unable to set osd_memory_target on smithi082 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T03:27:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:15 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-12T03:27:16.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:15 smithi142 ceph-mon[35329]: Unable to set osd_memory_target on smithi142 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:27:16.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:15 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:15 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: Unable to set osd_memory_target on smithi050 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: Updating smithi082:/etc/ceph/ceph.conf 2024-09-12T03:27:16.856 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:16 smithi082 ceph-mon[35046]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-12T03:27:16.859 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:16.860 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-12T03:27:16.860 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.860 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.861 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: Unable to set osd_memory_target on smithi050 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: Updating smithi082:/etc/ceph/ceph.conf 2024-09-12T03:27:16.862 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:16 smithi050 ceph-mon[30378]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: Unable to set osd_memory_target on smithi050 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-12T03:27:17.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: Updating smithi082:/etc/ceph/ceph.conf 2024-09-12T03:27:17.046 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:16 smithi142 ceph-mon[35329]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-12T03:27:18.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:17 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:17 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:27:18.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:18.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:17 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:18 smithi082 ceph-mon[35046]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:18 smithi082 ceph-mon[35046]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:18 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:18 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:18 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:18 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:18 smithi050 ceph-mon[30378]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:19.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:18 smithi050 ceph-mon[30378]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:19.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:18 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:19.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:18 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:19.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:18 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:19.277 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:18 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:19.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:18 smithi142 ceph-mon[35329]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:19.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:18 smithi142 ceph-mon[35329]: Updating smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:27:19.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:18 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:19.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:18 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:19.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:18 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:19.298 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:18 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:27:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:20 smithi082 ceph-mon[35046]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-09-12T03:27:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:20 smithi050 ceph-mon[30378]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-09-12T03:27:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:20 smithi142 ceph-mon[35329]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-09-12T03:27:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:22 smithi082 ceph-mon[35046]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-09-12T03:27:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:22 smithi050 ceph-mon[30378]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-09-12T03:27:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:22 smithi142 ceph-mon[35329]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-09-12T03:27:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:24 smithi082 ceph-mon[35046]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T03:27:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:24 smithi050 ceph-mon[30378]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T03:27:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:24 smithi142 ceph-mon[35329]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T03:27:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:26 smithi082 ceph-mon[35046]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:27:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:26 smithi050 ceph-mon[30378]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:27:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 -- 172.21.15.50:0/410732507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 msgr2=0x7f7c88101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/410732507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c88101d10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009930 tx=0x7f7c7402a420 comp rx=0 tx=0).stop 2024-09-12T03:27:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 -- 172.21.15.50:0/410732507 shutdown_connections 2024-09-12T03:27:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/410732507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102250 0x7f7c8810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/410732507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c88101d10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/410732507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c881078c0 0x7f7c88107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 -- 172.21.15.50:0/410732507 >> 172.21.15.50:0/410732507 conn(0x7f7c880fd5d0 msgr2=0x7f7c880ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 -- 172.21.15.50:0/410732507 shutdown_connections 2024-09-12T03:27:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.274+0000 7f7c8ca00640 1 -- 172.21.15.50:0/410732507 wait complete. 2024-09-12T03:27:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.275+0000 7f7c8ca00640 1 Processor -- start 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.275+0000 7f7c8ca00640 1 -- start start 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c8ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c8819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c8ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102250 0x7f7c8819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c8ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c881078c0 0x7f7c8819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c8ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111d20 con 0x7f7c881018b0 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c8ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111ba0 con 0x7f7c881078c0 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c8ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111ea0 con 0x7f7c88102250 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c86800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c8819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c86800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c8819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34606/0 (socket says 172.21.15.50:34606) 2024-09-12T03:27:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c86800640 1 -- 172.21.15.50:0/3300676395 learned_addr learned my addr 172.21.15.50:0/3300676395 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c87200640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c881078c0 0x7f7c8819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.276+0000 7f7c85e00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102250 0x7f7c8819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c86800640 1 -- 172.21.15.50:0/3300676395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102250 msgr2=0x7f7c8819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c86800640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102250 0x7f7c8819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c86800640 1 -- 172.21.15.50:0/3300676395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c881078c0 msgr2=0x7f7c8819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c86800640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c881078c0 0x7f7c8819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c86800640 1 -- 172.21.15.50:0/3300676395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8819f470 con 0x7f7c881018b0 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c86800640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c8819a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9f0 tx=0x7f7c7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c7f400640 1 -- 172.21.15.50:0/3300676395 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800cd80 con 0x7f7c881018b0 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8819f760 con 0x7f7c881018b0 2024-09-12T03:27:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c7f400640 1 -- 172.21.15.50:0/3300676395 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c78010910 con 0x7f7c881018b0 2024-09-12T03:27:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c7f400640 1 -- 172.21.15.50:0/3300676395 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780188f0 con 0x7f7c881018b0 2024-09-12T03:27:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.277+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c881a6ff0 con 0x7f7c881018b0 2024-09-12T03:27:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.280+0000 7f7c9046d740 1 Processor -- start 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.280+0000 7f7c7f400640 1 -- 172.21.15.50:0/3300676395 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f7c78010ac0 con 0x7f7c881018b0 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.280+0000 7f7c9046d740 1 -- start start 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e248790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e253030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e255960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c9046d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a63e1ae100 con 0x55a63e253570 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c9046d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a63e1561b0 con 0x55a63e2483b0 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c9046d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a63e1aa0c0 con 0x55a63e248cd0 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c85e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e255960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e248790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c85e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e255960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34612/0 (socket says 172.21.15.50:34612) 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e248790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59896/0 (socket says 172.21.15.50:59896) 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c86800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e253030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c85e00640 1 -- 172.21.15.50:0/246167102 learned_addr learned my addr 172.21.15.50:0/246167102 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 -- 172.21.15.50:0/246167102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 msgr2=0x55a63e253030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e253030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 -- 172.21.15.50:0/246167102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 msgr2=0x55a63e255960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e255960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.281+0000 7f7c87200640 1 -- 172.21.15.50:0/246167102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63dec1210 con 0x55a63e2483b0 2024-09-12T03:27:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c7f400640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7c6c0a2d80 0x7f7c6c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c87200640 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e248790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009f00 tx=0x7f7c80011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb2c51d1a8cdd3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c7f400640 1 -- 172.21.15.50:0/3300676395 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7c780cfd40 con 0x7f7c881018b0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c86800640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7c6c0a2d80 0x7f7c6c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c73400640 1 -- 172.21.15.50:0/246167102 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004d20 con 0x55a63e2483b0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c73400640 1 -- 172.21.15.50:0/246167102 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80004030 con 0x55a63e2483b0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.282+0000 7f7c73400640 1 -- 172.21.15.50:0/246167102 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80005260 con 0x55a63e2483b0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 -- 172.21.15.50:0/246167102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 msgr2=0x55a63e248790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c86800640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7c6c0a2d80 0x7f7c6c0a5240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7c780cd040 tx=0x7f7c780d3320 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e248790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009f00 tx=0x7f7c80011810 comp rx=0 tx=0).stop 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 -- 172.21.15.50:0/246167102 shutdown_connections 2024-09-12T03:27:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e255960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e253030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 --2- 172.21.15.50:0/246167102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e248790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 -- 172.21.15.50:0/246167102 >> 172.21.15.50:0/246167102 conn(0x55a63e244920 msgr2=0x55a63e244d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 -- 172.21.15.50:0/246167102 shutdown_connections 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.283+0000 7f7c9046d740 1 -- 172.21.15.50:0/246167102 wait complete. 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 -- start start 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 Processor -- start 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e250650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e250b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e2510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a63e255f90 con 0x55a63e253570 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a63e137090 con 0x55a63e2483b0 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a63e256110 con 0x55a63e248cd0 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c9046d740 10 client.0 _ll_register_callbacks cb 0x55a63e18bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c87200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e250650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c87200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e250650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59900/0 (socket says 172.21.15.50:59900) 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.284+0000 7f7c87200640 1 -- 172.21.15.50:0/3527244040 learned_addr learned my addr 172.21.15.50:0/3527244040 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c87200640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e2510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e250b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 -- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 msgr2=0x55a63e250650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e250650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 -- 172.21.15.50:0/3527244040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 msgr2=0x55a63e2510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e2510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 -- 172.21.15.50:0/3527244040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63df8d620 con 0x55a63e248cd0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c87200640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e250650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c85e00640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e250b90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c74006010 tx=0x7f7c740317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537d3eec3968921a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.285+0000 7f7c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c67e00640 1 -- 172.21.15.50:0/3527244040 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74035040 con 0x55a63e248cd0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c80004030 con 0x55a63e248cd0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c67e00640 1 -- 172.21.15.50:0/3527244040 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7c74005040 con 0x55a63e248cd0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c67e00640 1 -- 172.21.15.50:0/3527244040 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7c74005240 con 0x55a63e248cd0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c9046d740 10 client.25249 fetch_fsmap learned FSMap version 26 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a63e261e90 con 0x55a63e248cd0 2024-09-12T03:27:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c67e00640 1 -- 172.21.15.50:0/3527244040 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80004030 con 0x55a63e248cd0 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c67e00640 1 -- 172.21.15.50:0/3527244040 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7c7403d5c0 con 0x55a63e248cd0 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c9046d740 10 client.25249 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.286+0000 7f7c9046d740 10 client.25249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c9046d740 20 client.25249 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c9046d740 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2633a0 0x55a63e265790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c9046d740 4 client.25249 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a63e1bb830 con 0x55a63e2633a0 2024-09-12T03:27:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c87200640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2633a0 0x55a63e265790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:26 smithi142 ceph-mon[35329]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:27:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c87200640 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2633a0 0x55a63e265790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6b49f4d1df8684 server_cookie=b0393639b74bdb12 in_seq=0 out_seq=0 2024-09-12T03:27:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.287+0000 7f7c67e00640 10 client.25249 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.288+0000 7f7c67e00640 1 -- 172.21.15.50:0/3527244040 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a63e1bb830 con 0x55a63e2633a0 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.288+0000 7f7c67e00640 10 client.25249 handle_command_reply: tid=0 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.288+0000 7f7c9046d740 1 client.25249 shutdown 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 msgr2=0x55a63e250b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e250b90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c74006010 tx=0x7f7c740317e0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 shutdown_connections 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e2510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2633a0 0x55a63e265790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e250b90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 --2- 172.21.15.50:0/3527244040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e250650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 >> 172.21.15.50:0/3527244040 conn(0x55a63e244920 msgr2=0x55a63e24a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c67e00640 0 client.25249 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 shutdown_connections 2024-09-12T03:27:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 1 -- 172.21.15.50:0/3527244040 wait complete. 2024-09-12T03:27:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.289+0000 7f7c9046d740 20 client.25249 trim_cache size 0 max 16384 2024-09-12T03:27:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.304+0000 7f7c9046d740 1 Processor -- start 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.304+0000 7f7c9046d740 1 -- start start 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 0x55a63e24ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e248cd0 0x55a63e24f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e253570 0x55a63e26b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c9046d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881b3840 con 0x55a63e2483b0 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c9046d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881b1110 con 0x55a63e248cd0 2024-09-12T03:27:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c9046d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881aef60 con 0x55a63e253570 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 0x55a63e24ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 0x55a63e24ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34622/0 (socket says 172.21.15.50:34622) 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 -- 172.21.15.50:0/2705546137 learned_addr learned my addr 172.21.15.50:0/2705546137 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c85e00640 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e248cd0 0x55a63e24f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e253570 0x55a63e26b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 -- 172.21.15.50:0/2705546137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e253570 msgr2=0x55a63e26b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e253570 0x55a63e26b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 -- 172.21.15.50:0/2705546137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e248cd0 msgr2=0x55a63e24f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e248cd0 0x55a63e24f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.305+0000 7f7c87200640 1 -- 172.21.15.50:0/2705546137 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63e25f2d0 con 0x55a63e2483b0 2024-09-12T03:27:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.306+0000 7f7c87200640 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 0x55a63e24ef00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c80011d10 tx=0x7f7c80009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc694f0cda29bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.306+0000 7f7c72a00640 1 -- 172.21.15.50:0/2705546137 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8001e040 con 0x55a63e2483b0 2024-09-12T03:27:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.306+0000 7f7c72a00640 1 -- 172.21.15.50:0/2705546137 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80004e40 con 0x55a63e2483b0 2024-09-12T03:27:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.306+0000 7f7c72a00640 1 -- 172.21.15.50:0/2705546137 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800057e0 con 0x55a63e2483b0 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.306+0000 7f7c9046d740 1 -- 172.21.15.50:0/2705546137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 msgr2=0x55a63e24ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.306+0000 7f7c9046d740 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 0x55a63e24ef00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c80011d10 tx=0x7f7c80009500 comp rx=0 tx=0).stop 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 -- 172.21.15.50:0/2705546137 shutdown_connections 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e253570 0x55a63e26b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e248cd0 0x55a63e24f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 --2- 172.21.15.50:0/2705546137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e2483b0 0x55a63e24ef00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 -- 172.21.15.50:0/2705546137 >> 172.21.15.50:0/2705546137 conn(0x55a63e244920 msgr2=0x55a63e249550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 -- 172.21.15.50:0/2705546137 shutdown_connections 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 -- 172.21.15.50:0/2705546137 wait complete. 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 -- start start 2024-09-12T03:27:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.307+0000 7f7c9046d740 1 Processor -- start 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e24e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e25b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e25bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881af5d0 con 0x55a63e253570 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881b0a40 con 0x55a63e2483b0 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881afca0 con 0x55a63e248cd0 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c9046d740 10 client.0 _ll_register_callbacks cb 0x7f7c74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e25bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c85e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e25b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e25bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34632/0 (socket says 172.21.15.50:34632) 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 -- 172.21.15.50:0/488193208 learned_addr learned my addr 172.21.15.50:0/488193208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e24e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 -- 172.21.15.50:0/488193208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 msgr2=0x55a63e25b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e25b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 -- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 msgr2=0x55a63e24e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e24e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c87200640 1 -- 172.21.15.50:0/488193208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63e24c950 con 0x55a63e253570 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.308+0000 7f7c85e00640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e25b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c87200640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e25bbd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7c80011ee0 tx=0x7f7c8002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48f1ec72e2266b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c67e00640 1 -- 172.21.15.50:0/488193208 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800044f0 con 0x55a63e253570 2024-09-12T03:27:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c67e00640 1 -- 172.21.15.50:0/488193208 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7c80005000 con 0x55a63e253570 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a63e1b5b80 con 0x55a63e253570 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c67e00640 1 -- 172.21.15.50:0/488193208 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7c800055a0 con 0x55a63e253570 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c9046d740 10 client.15246 fetch_fsmap learned FSMap version 26 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a63e2cd3e0 con 0x55a63e253570 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.309+0000 7f7c67e00640 1 -- 172.21.15.50:0/488193208 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a63e1b5b80 con 0x55a63e253570 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c67e00640 1 -- 172.21.15.50:0/488193208 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7c80024040 con 0x55a63e253570 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c9046d740 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c9046d740 10 client.15246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c9046d740 20 client.15246 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c9046d740 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2cd8b0 0x55a63e2cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c9046d740 4 client.15246 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:27:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a63e1327c0 con 0x55a63e2cd8b0 2024-09-12T03:27:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.310+0000 7f7c85e00640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2cd8b0 0x55a63e2cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.311+0000 7f7c85e00640 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2cd8b0 0x55a63e2cfca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebea8bf73782da3 server_cookie=ac0e948eb23e9a5e in_seq=0 out_seq=0 2024-09-12T03:27:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.311+0000 7f7c67e00640 10 client.15246 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c67e00640 1 -- 172.21.15.50:0/488193208 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a63e1327c0 con 0x55a63e2cd8b0 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c67e00640 10 client.15246 handle_command_reply: tid=0 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 client.15246 shutdown 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 msgr2=0x55a63e25bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e25bbd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7c80011ee0 tx=0x7f7c8002b040 comp rx=0 tx=0).stop 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 shutdown_connections 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a63e253570 0x55a63e25bbd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a63e2cd8b0 0x55a63e2cfca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63e248cd0 0x55a63e25b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 --2- 172.21.15.50:0/488193208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a63e2483b0 0x55a63e24e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c67e00640 0 client.15246 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 >> 172.21.15.50:0/488193208 conn(0x55a63e244920 msgr2=0x55a63e2431a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 shutdown_connections 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 1 -- 172.21.15.50:0/488193208 wait complete. 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.323+0000 7f7c9046d740 20 client.15246 trim_cache size 0 max 16384 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:27:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:27:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7c6c0a2d80 msgr2=0x7f7c6c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7c6c0a2d80 0x7f7c6c0a5240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7c780cd040 tx=0x7f7c780d3320 comp rx=0 tx=0).stop 2024-09-12T03:27:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 msgr2=0x7f7c8819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c8819a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9f0 tx=0x7f7c7800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 shutdown_connections 2024-09-12T03:27:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c881078c0 0x7f7c8819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102250 0x7f7c8819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7c6c0a2d80 0x7f7c6c0a5240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 --2- 172.21.15.50:0/3300676395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c881018b0 0x7f7c8819a480 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 >> 172.21.15.50:0/3300676395 conn(0x7f7c880fd5d0 msgr2=0x7f7c8810cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 shutdown_connections 2024-09-12T03:27:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:27.324+0000 7f7c8ca00640 1 -- 172.21.15.50:0/3300676395 wait complete. 2024-09-12T03:27:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:28 smithi082 ceph-mon[35046]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:27:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:28 smithi050 ceph-mon[30378]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:27:29.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:28 smithi142 ceph-mon[35329]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:27:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:30 smithi082 ceph-mon[35046]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T03:27:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:30 smithi050 ceph-mon[30378]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T03:27:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:30 smithi142 ceph-mon[35329]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T03:27:33.096 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:32 smithi142 ceph-mon[35329]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:27:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:32 smithi082 ceph-mon[35046]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:27:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:32 smithi050 ceph-mon[30378]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:27:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:34 smithi082 ceph-mon[35046]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-12T03:27:35.168 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:34 smithi050 ceph-mon[30378]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-12T03:27:35.299 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:34 smithi142 ceph-mon[35329]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-12T03:27:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:36 smithi082 ceph-mon[35046]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:36 smithi050 ceph-mon[30378]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:36 smithi142 ceph-mon[35329]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:38 smithi082 ceph-mon[35046]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:38 smithi050 ceph-mon[30378]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:39.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:38 smithi142 ceph-mon[35329]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 -- 172.21.15.50:0/4017401991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd81018b0 msgr2=0x7f2bd8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 --2- 172.21.15.50:0/4017401991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd81018b0 0x7f2bd8101d10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009930 tx=0x7f2bc402a420 comp rx=0 tx=0).stop 2024-09-12T03:27:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 -- 172.21.15.50:0/4017401991 shutdown_connections 2024-09-12T03:27:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 --2- 172.21.15.50:0/4017401991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bd8102250 0x7f2bd810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 --2- 172.21.15.50:0/4017401991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd81018b0 0x7f2bd8101d10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 --2- 172.21.15.50:0/4017401991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd81078c0 0x7f2bd8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.448+0000 7f2bde200640 1 -- 172.21.15.50:0/4017401991 >> 172.21.15.50:0/4017401991 conn(0x7f2bd80fd5d0 msgr2=0x7f2bd80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.449+0000 7f2bde200640 1 -- 172.21.15.50:0/4017401991 shutdown_connections 2024-09-12T03:27:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.449+0000 7f2bde200640 1 -- 172.21.15.50:0/4017401991 wait complete. 2024-09-12T03:27:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 Processor -- start 2024-09-12T03:27:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 -- start start 2024-09-12T03:27:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bd81018b0 0x7f2bd819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 0x7f2bd819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd81078c0 0x7f2bd819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8111d00 con 0x7f2bd8102250 2024-09-12T03:27:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8111b80 con 0x7f2bd81018b0 2024-09-12T03:27:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bde200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8111e80 con 0x7f2bd81078c0 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 0x7f2bd819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bd81018b0 0x7f2bd819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 0x7f2bd819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37304/0 (socket says 172.21.15.50:37304) 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bd7e00640 1 -- 172.21.15.50:0/1566072022 learned_addr learned my addr 172.21.15.50:0/1566072022 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.450+0000 7f2bdd800640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd81078c0 0x7f2bd819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd7e00640 1 -- 172.21.15.50:0/1566072022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd81078c0 msgr2=0x7f2bd819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd81078c0 0x7f2bd819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd7e00640 1 -- 172.21.15.50:0/1566072022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bd81018b0 msgr2=0x7f2bd819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bd81018b0 0x7f2bd819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd7e00640 1 -- 172.21.15.50:0/1566072022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd819f460 con 0x7f2bd8102250 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 0x7f2bd819a9b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009230 tx=0x7f2bc4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.451+0000 7f2bd5600640 1 -- 172.21.15.50:0/1566072022 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc403f030 con 0x7f2bd8102250 2024-09-12T03:27:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.452+0000 7f2bd5600640 1 -- 172.21.15.50:0/1566072022 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4004440 con 0x7f2bd8102250 2024-09-12T03:27:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.452+0000 7f2bd5600640 1 -- 172.21.15.50:0/1566072022 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4035070 con 0x7f2bd8102250 2024-09-12T03:27:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.452+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd819f6f0 con 0x7f2bd8102250 2024-09-12T03:27:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.452+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a6fe0 con 0x7f2bd8102250 2024-09-12T03:27:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.455+0000 7f2be26d7740 1 Processor -- start 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.455+0000 7f2bd5600640 1 -- 172.21.15.50:0/1566072022 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2bc4042030 con 0x7f2bd8102250 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.455+0000 7f2be26d7740 1 -- start start 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbb8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2be26d7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a91cb1e100 con 0x55a91cbb83b0 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2be26d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a91cac61b0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2be26d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a91cb1a0c0 con 0x55a91cbc3570 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbb8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbb8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37316/0 (socket says 172.21.15.50:37316) 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 -- 172.21.15.50:0/641284466 learned_addr learned my addr 172.21.15.50:0/641284466 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdce00640 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 -- 172.21.15.50:0/641284466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 msgr2=0x55a91cbc5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 -- 172.21.15.50:0/641284466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 msgr2=0x55a91cbc3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.456+0000 7f2bdd800640 1 -- 172.21.15.50:0/641284466 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91c831210 con 0x55a91cbb83b0 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bdce00640 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bd5600640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f2bc00a2e00 0x7f2bc00a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bdd800640 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbb8790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009fd0 tx=0x7f2bd00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9f87ec32aec3af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bd5600640 1 -- 172.21.15.50:0/1566072022 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2bc40efc00 con 0x7f2bd8102250 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bdce00640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f2bc00a2e00 0x7f2bc00a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bca000640 1 -- 172.21.15.50:0/641284466 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0004d20 con 0x55a91cbb83b0 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bca000640 1 -- 172.21.15.50:0/641284466 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0004050 con 0x55a91cbb83b0 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.457+0000 7f2bca000640 1 -- 172.21.15.50:0/641284466 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0005260 con 0x55a91cbb83b0 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 -- 172.21.15.50:0/641284466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 msgr2=0x55a91cbb8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbb8790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009fd0 tx=0x7f2bd00118d0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2bdce00640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f2bc00a2e00 0x7f2bc00a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x55a91cbb5780 tx=0x7f2bcc005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 -- 172.21.15.50:0/641284466 shutdown_connections 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 --2- 172.21.15.50:0/641284466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbb8790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.458+0000 7f2be26d7740 1 -- 172.21.15.50:0/641284466 >> 172.21.15.50:0/641284466 conn(0x55a91cbb4920 msgr2=0x55a91cbb4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.459+0000 7f2be26d7740 1 -- 172.21.15.50:0/641284466 shutdown_connections 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.459+0000 7f2be26d7740 1 -- 172.21.15.50:0/641284466 wait complete. 2024-09-12T03:27:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.459+0000 7f2be26d7740 1 -- start start 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.459+0000 7f2be26d7740 1 Processor -- start 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbb83b0 0x55a91cbc0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbc3570 0x55a91cbc10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a91cbc5f90 con 0x55a91cbb8cd0 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a91caa7090 con 0x55a91cbc3570 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a91cbc6110 con 0x55a91cbb83b0 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2be26d7740 10 client.0 _ll_register_callbacks cb 0x55a91cafbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37320/0 (socket says 172.21.15.50:37320) 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbb83b0 0x55a91cbc0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2bd7e00640 1 -- 172.21.15.50:0/3710273304 learned_addr learned my addr 172.21.15.50:0/3710273304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.460+0000 7f2bdd800640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbc3570 0x55a91cbc10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bd7e00640 1 -- 172.21.15.50:0/3710273304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbb83b0 msgr2=0x55a91cbc0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbb83b0 0x55a91cbc0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bd7e00640 1 -- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbc3570 msgr2=0x55a91cbc10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbc3570 0x55a91cbc10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bd7e00640 1 -- 172.21.15.50:0/3710273304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91c8fd620 con 0x55a91cbb8cd0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bdd800640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbc3570 0x55a91cbc10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc0b60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009a50 tx=0x7f2bc4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199ca9fa9244add5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bbea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bbea00640 1 -- 172.21.15.50:0/3710273304 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4049070 con 0x55a91cbb8cd0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bbea00640 1 -- 172.21.15.50:0/3710273304 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2bc4031bd0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd0004050 con 0x55a91cbb8cd0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2bbea00640 1 -- 172.21.15.50:0/3710273304 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2bc40b7f70 con 0x55a91cbb8cd0 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2be26d7740 10 client.15258 fetch_fsmap learned FSMap version 26 2024-09-12T03:27:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.461+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a91cbd1dc0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2bbea00640 1 -- 172.21.15.50:0/3710273304 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0004050 con 0x55a91cbb8cd0 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2bbea00640 1 -- 172.21.15.50:0/3710273304 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2bc40f7040 con 0x55a91cbb8cd0 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2be26d7740 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2be26d7740 10 client.15258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2be26d7740 20 client.15258 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cbd3270 0x55a91cbd5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2be26d7740 4 client.15258 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.462+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a91cb2b830 con 0x55a91cbd3270 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.463+0000 7f2bdce00640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cbd3270 0x55a91cbd5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.463+0000 7f2bdce00640 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cbd3270 0x55a91cbd5660 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea720b46a0c0187f server_cookie=2807f6df25a225d2 in_seq=0 out_seq=0 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.463+0000 7f2bbea00640 10 client.15258 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2bbea00640 1 -- 172.21.15.50:0/3710273304 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a91cb2b830 con 0x55a91cbd3270 2024-09-12T03:27:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2bbea00640 10 client.15258 handle_command_reply: tid=0 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 client.15258 shutdown 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 msgr2=0x55a91cbc0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc0b60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009a50 tx=0x7f2bc4005b50 comp rx=0 tx=0).stop 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 shutdown_connections 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbc3570 0x55a91cbc10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc0b60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cbd3270 0x55a91cbd5660 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3710273304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbb83b0 0x55a91cbc0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 >> 172.21.15.50:0/3710273304 conn(0x55a91cbb4920 msgr2=0x55a91cbba680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2bbea00640 0 client.15258 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 shutdown_connections 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 1 -- 172.21.15.50:0/3710273304 wait complete. 2024-09-12T03:27:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.464+0000 7f2be26d7740 20 client.15258 trim_cache size 0 max 16384 2024-09-12T03:27:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 Processor -- start 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 -- start start 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbdaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbdb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbb9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b3970 con 0x55a91cbb83b0 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b1240 con 0x55a91cbb8cd0 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.480+0000 7f2be26d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81af090 con 0x55a91cbc3570 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbdaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbdaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37332/0 (socket says 172.21.15.50:37332) 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 -- 172.21.15.50:0/3474413352 learned_addr learned my addr 172.21.15.50:0/3474413352 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbdb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdd800640 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbb9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 -- 172.21.15.50:0/3474413352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 msgr2=0x55a91cbb9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbb9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 -- 172.21.15.50:0/3474413352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 msgr2=0x55a91cbdb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbdb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 -- 172.21.15.50:0/3474413352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91cbb55b0 con 0x55a91cbb83b0 2024-09-12T03:27:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdd800640 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbb9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.481+0000 7f2bdce00640 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbdaf30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a91cbd5ea0 tx=0x7f2bcc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd50be384ae3de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2bc9600640 1 -- 172.21.15.50:0/3474413352 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc005020 con 0x55a91cbb83b0 2024-09-12T03:27:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2bc9600640 1 -- 172.21.15.50:0/3474413352 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc0047a0 con 0x55a91cbb83b0 2024-09-12T03:27:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2bc9600640 1 -- 172.21.15.50:0/3474413352 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc02b680 con 0x55a91cbb83b0 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 -- 172.21.15.50:0/3474413352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 msgr2=0x55a91cbdaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbdaf30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a91cbd5ea0 tx=0x7f2bcc01b000 comp rx=0 tx=0).stop 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 -- 172.21.15.50:0/3474413352 shutdown_connections 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbb9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbdb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 --2- 172.21.15.50:0/3474413352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbdaf30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.482+0000 7f2be26d7740 1 -- 172.21.15.50:0/3474413352 >> 172.21.15.50:0/3474413352 conn(0x55a91cbb4920 msgr2=0x55a91cbbeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.483+0000 7f2be26d7740 1 -- 172.21.15.50:0/3474413352 shutdown_connections 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.483+0000 7f2be26d7740 1 -- 172.21.15.50:0/3474413352 wait complete. 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.483+0000 7f2be26d7740 1 -- start start 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.483+0000 7f2be26d7740 1 Processor -- start 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbc6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81af700 con 0x55a91cbb83b0 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b0b70 con 0x55a91cbb8cd0 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81afdd0 con 0x55a91cbc3570 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2be26d7740 10 client.0 _ll_register_callbacks cb 0x7f2bc4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbc6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2bdce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbc6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37344/0 (socket says 172.21.15.50:37344) 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2bdce00640 1 -- 172.21.15.50:0/2120528733 learned_addr learned my addr 172.21.15.50:0/2120528733 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2bdd800640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.484+0000 7f2bd7e00640 1 -- 172.21.15.50:0/2120528733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 msgr2=0x55a91cbc6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bd7e00640 1 -- 172.21.15.50:0/2120528733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 msgr2=0x55a91cbc6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbc6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bd7e00640 1 -- 172.21.15.50:0/2120528733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91cbc15f0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bdce00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bdd800640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bd7e00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc6940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55a91cbc1bd0 tx=0x7f2bc40ecfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dc7cbfd195152f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bbea00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bbea00640 1 -- 172.21.15.50:0/2120528733 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4049040 con 0x55a91cbb8cd0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bcc0047a0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bbea00640 1 -- 172.21.15.50:0/2120528733 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2bc40bb320 con 0x55a91cbb8cd0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2bbea00640 1 -- 172.21.15.50:0/2120528733 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2bc40a2050 con 0x55a91cbb8cd0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2be26d7740 10 client.25283 fetch_fsmap learned FSMap version 26 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.485+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a91cc3d130 con 0x55a91cbb8cd0 2024-09-12T03:27:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2bbea00640 1 -- 172.21.15.50:0/2120528733 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc0047a0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2bbea00640 1 -- 172.21.15.50:0/2120528733 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2bc40c4ae0 con 0x55a91cbb8cd0 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2be26d7740 10 client.25283 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2be26d7740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2be26d7740 20 client.25283 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2be26d7740 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cc3d720 0x55a91cc3fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2be26d7740 4 client.25283 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a91cb25b80 con 0x55a91cc3d720 2024-09-12T03:27:40.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.486+0000 7f2bdce00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cc3d720 0x55a91cc3fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.487+0000 7f2bdce00640 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cc3d720 0x55a91cc3fb40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d19eb0b2cc53354 server_cookie=a0fe2e749b41bd81 in_seq=0 out_seq=0 2024-09-12T03:27:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.487+0000 7f2bbea00640 10 client.25283 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.490+0000 7f2bbea00640 1 -- 172.21.15.50:0/2120528733 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a91cb25b80 con 0x55a91cc3d720 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.490+0000 7f2bbea00640 10 client.25283 handle_command_reply: tid=0 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 client.25283 shutdown 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 msgr2=0x55a91cbc6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc6940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55a91cbc1bd0 tx=0x7f2bc40ecfe0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 shutdown_connections 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a91cc3d720 0x55a91cc3fb40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a91cbc3570 0x55a91cbc6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91cbb8cd0 0x55a91cbc6940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2bbea00640 0 client.25283 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 --2- 172.21.15.50:0/2120528733 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a91cbb83b0 0x55a91cbc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 >> 172.21.15.50:0/2120528733 conn(0x55a91cbb4920 msgr2=0x55a91cbc05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 shutdown_connections 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 1 -- 172.21.15.50:0/2120528733 wait complete. 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.491+0000 7f2be26d7740 20 client.25283 trim_cache size 0 max 16384 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:27:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:27:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f2bc00a2e00 msgr2=0x7f2bc00a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f2bc00a2e00 0x7f2bc00a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x55a91cbb5780 tx=0x7f2bcc005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 msgr2=0x7f2bd819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 0x7f2bd819a9b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009230 tx=0x7f2bc4004040 comp rx=0 tx=0).stop 2024-09-12T03:27:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 shutdown_connections 2024-09-12T03:27:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd81078c0 0x7f2bd819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f2bc00a2e00 0x7f2bc00a52c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bd8102250 0x7f2bd819a9b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 --2- 172.21.15.50:0/1566072022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bd81018b0 0x7f2bd819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 >> 172.21.15.50:0/1566072022 conn(0x7f2bd80fd5d0 msgr2=0x7f2bd80fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 shutdown_connections 2024-09-12T03:27:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:40.492+0000 7f2bde200640 1 -- 172.21.15.50:0/1566072022 wait complete. 2024-09-12T03:27:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:40 smithi082 ceph-mon[35046]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-12T03:27:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:40 smithi050 ceph-mon[30378]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-12T03:27:41.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:40 smithi142 ceph-mon[35329]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-12T03:27:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:42 smithi082 ceph-mon[35046]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:42 smithi050 ceph-mon[30378]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:43.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:42 smithi142 ceph-mon[35329]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-12T03:27:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:44 smithi082 ceph-mon[35046]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-12T03:27:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:44 smithi050 ceph-mon[30378]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-12T03:27:45.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:44 smithi142 ceph-mon[35329]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-12T03:27:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:46 smithi082 ceph-mon[35046]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:47.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:46 smithi050 ceph-mon[30378]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:47.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:46 smithi142 ceph-mon[35329]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:48 smithi082 ceph-mon[35046]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:48 smithi050 ceph-mon[30378]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:49.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:48 smithi142 ceph-mon[35329]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:50 smithi082 ceph-mon[35046]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:27:51.172 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:50 smithi050 ceph-mon[30378]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:27:51.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:50 smithi142 ceph-mon[35329]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:27:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:52 smithi050 ceph-mon[30378]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:53.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:52 smithi142 ceph-mon[35329]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:52 smithi082 ceph-mon[35046]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:27:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.612+0000 7fe73ae00640 1 -- 172.21.15.50:0/3353506466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73410e170 msgr2=0x7fe734110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.612+0000 7fe73ae00640 1 --2- 172.21.15.50:0/3353506466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73410e170 0x7fe734110560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009be0 tx=0x7fe72c02a750 comp rx=0 tx=0).stop 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.612+0000 7fe73ae00640 1 -- 172.21.15.50:0/3353506466 shutdown_connections 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.612+0000 7fe73ae00640 1 --2- 172.21.15.50:0/3353506466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73410e170 0x7fe734110560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.613+0000 7fe73ae00640 1 --2- 172.21.15.50:0/3353506466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe734074ec0 0x7fe73410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.613+0000 7fe73ae00640 1 --2- 172.21.15.50:0/3353506466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe734074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.613+0000 7fe73ae00640 1 -- 172.21.15.50:0/3353506466 >> 172.21.15.50:0/3353506466 conn(0x7fe7340fd5d0 msgr2=0x7fe7340ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.613+0000 7fe73ae00640 1 -- 172.21.15.50:0/3353506466 shutdown_connections 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.613+0000 7fe73ae00640 1 -- 172.21.15.50:0/3353506466 wait complete. 2024-09-12T03:27:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.614+0000 7fe73ae00640 1 Processor -- start 2024-09-12T03:27:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.614+0000 7fe73ae00640 1 -- start start 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe73ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe73419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe73ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe734074ec0 0x7fe73419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe73ae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe73410e170 0x7fe73419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe73ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe734113e60 con 0x7fe7340745a0 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe73ae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe734113ce0 con 0x7fe73410e170 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe73ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe734113fe0 con 0x7fe734074ec0 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe739a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe73419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe739a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe73419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36646/0 (socket says 172.21.15.50:36646) 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe739a00640 1 -- 172.21.15.50:0/459232073 learned_addr learned my addr 172.21.15.50:0/459232073 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.615+0000 7fe739000640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe734074ec0 0x7fe73419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe739a00640 1 -- 172.21.15.50:0/459232073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe734074ec0 msgr2=0x7fe73419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe739a00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe734074ec0 0x7fe73419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe73a400640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe73410e170 0x7fe73419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe739a00640 1 -- 172.21.15.50:0/459232073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe73410e170 msgr2=0x7fe73419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe739a00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe73410e170 0x7fe73419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe739a00640 1 -- 172.21.15.50:0/459232073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73419f3f0 con 0x7fe7340745a0 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe73a400640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe73410e170 0x7fe73419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe739a00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe73419a400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe72800e9c0 tx=0x7fe72800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe732000640 1 -- 172.21.15.50:0/459232073 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72800ccc0 con 0x7fe7340745a0 2024-09-12T03:27:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.616+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73419f6e0 con 0x7fe7340745a0 2024-09-12T03:27:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.617+0000 7fe732000640 1 -- 172.21.15.50:0/459232073 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe728005500 con 0x7fe7340745a0 2024-09-12T03:27:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.617+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7341a6f70 con 0x7fe7340745a0 2024-09-12T03:27:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.617+0000 7fe732000640 1 -- 172.21.15.50:0/459232073 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728018430 con 0x7fe7340745a0 2024-09-12T03:27:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.621+0000 7fe73f20a740 1 Processor -- start 2024-09-12T03:27:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 -- start start 2024-09-12T03:27:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe732000640 1 -- 172.21.15.50:0/459232073 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fe7280046c0 con 0x7fe7340745a0 2024-09-12T03:27:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e30fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e3b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e3e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f6c4d8e100 con 0x55f6c4e3bd80 2024-09-12T03:27:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f6c4d361b0 con 0x55f6c4e314e0 2024-09-12T03:27:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.622+0000 7fe73f20a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f6c4d8a0c0 con 0x55f6c4e30bc0 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe739a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e3b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e30fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe739000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e3e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e30fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:49596/0 (socket says 172.21.15.50:49596) 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 -- 172.21.15.50:0/261175349 learned_addr learned my addr 172.21.15.50:0/261175349 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 -- 172.21.15.50:0/261175349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 msgr2=0x55f6c4e3b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e3b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 -- 172.21.15.50:0/261175349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 msgr2=0x55f6c4e3e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e3e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.623+0000 7fe73a400640 1 -- 172.21.15.50:0/261175349 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c4aa1210 con 0x55f6c4e30bc0 2024-09-12T03:27:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.624+0000 7fe739000640 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.624+0000 7fe73a400640 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e30fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe72c02acc0 tx=0x7fe72c030ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf40fcc7aa268d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.624+0000 7fe732000640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fe71c0a2dd0 0x7fe71c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.624+0000 7fe732000640 1 -- 172.21.15.50:0/459232073 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe7280cff20 con 0x7fe7340745a0 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.624+0000 7fe726a00640 1 -- 172.21.15.50:0/261175349 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c041020 con 0x55f6c4e30bc0 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.625+0000 7fe726a00640 1 -- 172.21.15.50:0/261175349 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c037040 con 0x55f6c4e30bc0 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.625+0000 7fe726a00640 1 -- 172.21.15.50:0/261175349 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c036070 con 0x55f6c4e30bc0 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.625+0000 7fe739a00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fe71c0a2dd0 0x7fe71c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.625+0000 7fe73f20a740 1 -- 172.21.15.50:0/261175349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 msgr2=0x55f6c4e30fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e30fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe72c02acc0 tx=0x7fe72c030ac0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe739a00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fe71c0a2dd0 0x7fe71c0a5290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7280cd220 tx=0x7fe7280bc040 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 -- 172.21.15.50:0/261175349 shutdown_connections 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e3b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 --2- 172.21.15.50:0/261175349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e30fa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 -- 172.21.15.50:0/261175349 >> 172.21.15.50:0/261175349 conn(0x55f6c4e2d130 msgr2=0x55f6c4e2d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 -- 172.21.15.50:0/261175349 shutdown_connections 2024-09-12T03:27:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 -- 172.21.15.50:0/261175349 wait complete. 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.626+0000 7fe73f20a740 1 -- start start 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 Processor -- start 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f6c4e3e950 con 0x55f6c4e3bd80 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f6c4d17090 con 0x55f6c4e30bc0 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f6c4e3ead0 con 0x55f6c4e314e0 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.627+0000 7fe73f20a740 10 client.0 _ll_register_callbacks cb 0x55f6c4d6bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:54680/0 (socket says 172.21.15.50:54680) 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 -- 172.21.15.50:0/374718785 learned_addr learned my addr 172.21.15.50:0/374718785 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe739000640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 -- 172.21.15.50:0/374718785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 msgr2=0x55f6c4e391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e391a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 -- 172.21.15.50:0/374718785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 msgr2=0x55f6c4e396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe73a400640 1 -- 172.21.15.50:0/374718785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c4b6d620 con 0x55f6c4e30bc0 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.628+0000 7fe739000640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.629+0000 7fe73a400640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e38c60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe72c003b60 tx=0x7fe72c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a37f557f6bab90c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.629+0000 7fe71b400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.629+0000 7fe71b400640 1 -- 172.21.15.50:0/374718785 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c050060 con 0x55f6c4e30bc0 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.629+0000 7fe71b400640 1 -- 172.21.15.50:0/374718785 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe72c0305b0 con 0x55f6c4e30bc0 2024-09-12T03:27:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.629+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c037040 con 0x55f6c4e30bc0 2024-09-12T03:27:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.630+0000 7fe71b400640 1 -- 172.21.15.50:0/374718785 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe72c041030 con 0x55f6c4e30bc0 2024-09-12T03:27:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.630+0000 7fe73f20a740 10 client.25292 fetch_fsmap learned FSMap version 26 2024-09-12T03:27:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.630+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6c4e4a5d0 con 0x55f6c4e30bc0 2024-09-12T03:27:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.630+0000 7fe71b400640 1 -- 172.21.15.50:0/374718785 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c037040 con 0x55f6c4e30bc0 2024-09-12T03:27:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe71b400640 1 -- 172.21.15.50:0/374718785 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe72c03d510 con 0x55f6c4e30bc0 2024-09-12T03:27:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe73f20a740 10 client.25292 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:27:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe73f20a740 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe73f20a740 20 client.25292 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe73f20a740 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4e4b920 0x55f6c4e4dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe73f20a740 4 client.25292 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.631+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c4d9b830 con 0x55f6c4e4b920 2024-09-12T03:27:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.632+0000 7fe739000640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4e4b920 0x55f6c4e4dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.633+0000 7fe739000640 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4e4b920 0x55f6c4e4dd10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc475b03705c5f2 server_cookie=a07781c789f74bf5 in_seq=0 out_seq=0 2024-09-12T03:27:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.633+0000 7fe71b400640 10 client.25292 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.633+0000 7fe71b400640 1 -- 172.21.15.50:0/374718785 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6c4d9b830 con 0x55f6c4e4b920 2024-09-12T03:27:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.633+0000 7fe71b400640 10 client.25292 handle_command_reply: tid=0 2024-09-12T03:27:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.633+0000 7fe73f20a740 1 client.25292 shutdown 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 msgr2=0x55f6c4e38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e38c60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe72c003b60 tx=0x7fe72c003b90 comp rx=0 tx=0).stop 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 shutdown_connections 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4e4b920 0x55f6c4e4dd10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 --2- 172.21.15.50:0/374718785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e38c60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 >> 172.21.15.50:0/374718785 conn(0x55f6c4e2d130 msgr2=0x55f6c4e239e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe71b400640 0 client.25292 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 shutdown_connections 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 1 -- 172.21.15.50:0/374718785 wait complete. 2024-09-12T03:27:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.634+0000 7fe73f20a740 20 client.25292 trim_cache size 0 max 16384 2024-09-12T03:27:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 Processor -- start 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 -- start start 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e1f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e35ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341af4b0 con 0x55f6c4e30bc0 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341acd80 con 0x55f6c4e314e0 2024-09-12T03:27:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73f20a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341aabf0 con 0x55f6c4e3bd80 2024-09-12T03:27:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.650+0000 7fe73a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e1f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e35ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe73a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e1f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36686/0 (socket says 172.21.15.50:36686) 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe73a400640 1 -- 172.21.15.50:0/1209510521 learned_addr learned my addr 172.21.15.50:0/1209510521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe73a400640 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 -- 172.21.15.50:0/1209510521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e3bd80 msgr2=0x55f6c4e363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 -- 172.21.15.50:0/1209510521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 msgr2=0x55f6c4e1f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e1f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 -- 172.21.15.50:0/1209510521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c4e346d0 con 0x55f6c4e314e0 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe73a400640 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.651+0000 7fe739000640 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e35ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55f6c4e3a2a0 tx=0x7fe720005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b89fd9e12accfd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.652+0000 7fe726000640 1 -- 172.21.15.50:0/1209510521 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72002b070 con 0x55f6c4e314e0 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.652+0000 7fe726000640 1 -- 172.21.15.50:0/1209510521 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72001fba0 con 0x55f6c4e314e0 2024-09-12T03:27:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.652+0000 7fe726000640 1 -- 172.21.15.50:0/1209510521 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720026d70 con 0x55f6c4e314e0 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.652+0000 7fe73f20a740 1 -- 172.21.15.50:0/1209510521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 msgr2=0x55f6c4e35ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.652+0000 7fe73f20a740 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e35ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55f6c4e3a2a0 tx=0x7fe720005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 -- 172.21.15.50:0/1209510521 shutdown_connections 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e314e0 0x55f6c4e35ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 --2- 172.21.15.50:0/1209510521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e1f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 -- 172.21.15.50:0/1209510521 >> 172.21.15.50:0/1209510521 conn(0x55f6c4e2d130 msgr2=0x55f6c4e33fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 -- 172.21.15.50:0/1209510521 shutdown_connections 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 -- 172.21.15.50:0/1209510521 wait complete. 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 -- start start 2024-09-12T03:27:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.653+0000 7fe73f20a740 1 Processor -- start 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e50f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e4e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e4e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341ab240 con 0x55f6c4e30bc0 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341ac6b0 con 0x55f6c4e3bd80 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341ab910 con 0x55f6c4e314e0 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73f20a740 10 client.0 _ll_register_callbacks cb 0x7fe720009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe739000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e4e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe739000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e4e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:49630/0 (socket says 172.21.15.50:49630) 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe739000640 1 -- 172.21.15.50:0/4226568661 learned_addr learned my addr 172.21.15.50:0/4226568661 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:27:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.654+0000 7fe73a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e50f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe739000640 1 -- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e3bd80 msgr2=0x55f6c4e4e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe739000640 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e4e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe739000640 1 -- 172.21.15.50:0/4226568661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 msgr2=0x55f6c4e50f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe739000640 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e50f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe739000640 1 -- 172.21.15.50:0/4226568661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c4e4ee70 con 0x55f6c4e314e0 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe739000640 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e4e290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe720012470 tx=0x7fe720026510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=368b4c2b0c3ec0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe71b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe71b400640 1 -- 172.21.15.50:0/4226568661 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72002b040 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72001fba0 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.655+0000 7fe71b400640 1 -- 172.21.15.50:0/4226568661 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe720026bf0 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.656+0000 7fe71b400640 1 -- 172.21.15.50:0/4226568661 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe72003c050 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.656+0000 7fe73f20a740 10 client.25288 fetch_fsmap learned FSMap version 26 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.656+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6c4eb6070 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.656+0000 7fe71b400640 1 -- 172.21.15.50:0/4226568661 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72001fba0 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe71b400640 1 -- 172.21.15.50:0/4226568661 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe7200046c0 con 0x55f6c4e314e0 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73f20a740 10 client.25288 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73f20a740 10 client.25288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73f20a740 20 client.25288 populate_metadata read hostname 'smithi050' 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73f20a740 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4eb6770 0x55f6c4eb8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73f20a740 4 client.25288 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:27:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6c4d95b80 con 0x55f6c4eb6770 2024-09-12T03:27:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.657+0000 7fe73a400640 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4eb6770 0x55f6c4eb8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.658+0000 7fe73a400640 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4eb6770 0x55f6c4eb8b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19e205e22492ebc server_cookie=e23a245e256e575c in_seq=0 out_seq=0 2024-09-12T03:27:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.658+0000 7fe71b400640 10 client.25288 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.665+0000 7fe71b400640 1 -- 172.21.15.50:0/4226568661 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6c4d95b80 con 0x55f6c4eb6770 2024-09-12T03:27:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.665+0000 7fe71b400640 10 client.25288 handle_command_reply: tid=0 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.665+0000 7fe73f20a740 1 client.25288 shutdown 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 msgr2=0x55f6c4e4e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e4e290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe720012470 tx=0x7fe720026510 comp rx=0 tx=0).stop 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 shutdown_connections 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6c4e3bd80 0x55f6c4e4e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f6c4eb6770 0x55f6c4eb8b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6c4e314e0 0x55f6c4e4e290 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 --2- 172.21.15.50:0/4226568661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f6c4e30bc0 0x55f6c4e50f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 >> 172.21.15.50:0/4226568661 conn(0x55f6c4e2d130 msgr2=0x55f6c4e46d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe71b400640 0 client.25288 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 shutdown_connections 2024-09-12T03:27:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 1 -- 172.21.15.50:0/4226568661 wait complete. 2024-09-12T03:27:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.666+0000 7fe73f20a740 20 client.25288 trim_cache size 0 max 16384 2024-09-12T03:27:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:27:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:27:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:27:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fe71c0a2dd0 msgr2=0x7fe71c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fe71c0a2dd0 0x7fe71c0a5290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7280cd220 tx=0x7fe7280bc040 comp rx=0 tx=0).stop 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 msgr2=0x7fe73419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe73419a400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe72800e9c0 tx=0x7fe72800ee80 comp rx=0 tx=0).stop 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 shutdown_connections 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe73410e170 0x7fe73419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe734074ec0 0x7fe73419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fe71c0a2dd0 0x7fe71c0a5290 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 --2- 172.21.15.50:0/459232073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7340745a0 0x7fe73419a400 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.667+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 >> 172.21.15.50:0/459232073 conn(0x7fe7340fd5d0 msgr2=0x7fe7340fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.668+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 shutdown_connections 2024-09-12T03:27:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:27:53.668+0000 7fe73ae00640 1 -- 172.21.15.50:0/459232073 wait complete. 2024-09-12T03:27:55.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:54 smithi050 ceph-mon[30378]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T03:27:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:54 smithi142 ceph-mon[35329]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T03:27:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:54 smithi082 ceph-mon[35046]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T03:27:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:56 smithi050 ceph-mon[30378]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:27:57.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:56 smithi142 ceph-mon[35329]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:27:57.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:56 smithi082 ceph-mon[35046]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:27:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:27:58 smithi050 ceph-mon[30378]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:27:59.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:27:58 smithi142 ceph-mon[35329]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:27:59.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:27:58 smithi082 ceph-mon[35046]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:28:00.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:28:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:28:00.112+0000 7fd97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:28:00.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:28:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:28:00.113+0000 7fd97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:28:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:28:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:28:00.113+0000 7fd97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:28:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:28:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:28:00.113+0000 7fd97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:28:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:28:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:28:00.113+0000 7fd97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:28:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:00 smithi050 ceph-mon[30378]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T03:28:01.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:00 smithi142 ceph-mon[35329]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T03:28:01.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:00 smithi082 ceph-mon[35046]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-12T03:28:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:02 smithi050 ceph-mon[30378]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:28:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:02 smithi050 ceph-mon[30378]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-09-12T03:28:03.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:02 smithi142 ceph-mon[35329]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:28:03.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:02 smithi142 ceph-mon[35329]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-09-12T03:28:03.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:02 smithi082 ceph-mon[35046]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:28:03.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:02 smithi082 ceph-mon[35046]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-09-12T03:28:05.155 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:04 smithi082 ceph-mon[35046]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-12T03:28:05.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:04 smithi050 ceph-mon[30378]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-12T03:28:05.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:04 smithi142 ceph-mon[35329]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-12T03:28:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3368359413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a60102ae0 msgr2=0x7f7a60102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3368359413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a60102ae0 0x7f7a60102ec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009900 tx=0x7f7a5002a480 comp rx=0 tx=0).stop 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3368359413 shutdown_connections 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3368359413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a6010bf00 0x7f7a6010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3368359413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60103400 0x7f7a6010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3368359413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a60102ae0 0x7f7a60102ec0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3368359413 >> 172.21.15.50:0/3368359413 conn(0x7f7a600777c0 msgr2=0x7f7a600feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3368359413 shutdown_connections 2024-09-12T03:28:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.793+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3368359413 wait complete. 2024-09-12T03:28:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.794+0000 7f7a5fe00640 1 Processor -- start 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.794+0000 7f7a5fe00640 1 -- start start 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60102ae0 0x7f7a6019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a60103400 0x7f7a6019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 0x7f7a6019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60111d20 con 0x7f7a60103400 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60111ba0 con 0x7f7a60102ae0 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60111ea0 con 0x7f7a6010bf00 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 0x7f7a6019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 0x7f7a6019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58198/0 (socket says 172.21.15.50:58198) 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5f400640 1 -- 172.21.15.50:0/3089912945 learned_addr learned my addr 172.21.15.50:0/3089912945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60102ae0 0x7f7a6019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.795+0000 7f7a5e000640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a60103400 0x7f7a6019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5f400640 1 -- 172.21.15.50:0/3089912945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60102ae0 msgr2=0x7f7a6019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5f400640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60102ae0 0x7f7a6019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5f400640 1 -- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a60103400 msgr2=0x7f7a6019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5f400640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a60103400 0x7f7a6019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5f400640 1 -- 172.21.15.50:0/3089912945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6019f490 con 0x7f7a6010bf00 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60102ae0 0x7f7a6019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5f400640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 0x7f7a6019ed60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800eb10 tx=0x7f7a5800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a64a00640 1 -- 172.21.15.50:0/3089912945 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58004b20 con 0x7f7a6010bf00 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.796+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6019f780 con 0x7f7a6010bf00 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.797+0000 7f7a64a00640 1 -- 172.21.15.50:0/3089912945 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58004cc0 con 0x7f7a6010bf00 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.797+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601a7010 con 0x7f7a6010bf00 2024-09-12T03:28:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.797+0000 7f7a64a00640 1 -- 172.21.15.50:0/3089912945 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58010690 con 0x7f7a6010bf00 2024-09-12T03:28:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.801+0000 7f7a68df5740 1 Processor -- start 2024-09-12T03:28:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.801+0000 7f7a64a00640 1 -- 172.21.15.50:0/3089912945 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f7a58005000 con 0x7f7a6010bf00 2024-09-12T03:28:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.801+0000 7f7a68df5740 1 -- start start 2024-09-12T03:28:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9035030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c9037960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a68df5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567c8f90100 con 0x5567c902acd0 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a68df5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5567c8f381b0 con 0x5567c9035570 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a68df5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5567c8f8c0c0 con 0x5567c902a3b0 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a5e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9035030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9035030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46140/0 (socket says 172.21.15.50:46140) 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a5e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58214/0 (socket says 172.21.15.50:58214) 2024-09-12T03:28:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a5ea00640 1 -- 172.21.15.50:0/3575085552 learned_addr learned my addr 172.21.15.50:0/3575085552 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.802+0000 7f7a5e000640 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c9037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a5e000640 1 -- 172.21.15.50:0/3575085552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 msgr2=0x5567c9037960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a5e000640 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c9037960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a5e000640 1 -- 172.21.15.50:0/3575085552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 msgr2=0x5567c9035030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a5e000640 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9035030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a5e000640 1 -- 172.21.15.50:0/3575085552 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c8ca3210 con 0x5567c902a3b0 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a5e000640 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902a790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009730 tx=0x7f7a4c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14779f5ca120bce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.803+0000 7f7a4be00640 1 -- 172.21.15.50:0/3575085552 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c004050 con 0x5567c902a3b0 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a4be00640 1 -- 172.21.15.50:0/3575085552 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c005500 con 0x5567c902a3b0 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a4be00640 1 -- 172.21.15.50:0/3575085552 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c004d90 con 0x5567c902a3b0 2024-09-12T03:28:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a64a00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7a440a2e90 0x7f7a440a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a64a00640 1 -- 172.21.15.50:0/3089912945 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7a580cff40 con 0x7f7a6010bf00 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7a440a2e90 0x7f7a440a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a68df5740 1 -- 172.21.15.50:0/3575085552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 msgr2=0x5567c902a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.804+0000 7f7a68df5740 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902a790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009730 tx=0x7f7a4c0096c0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 -- 172.21.15.50:0/3575085552 shutdown_connections 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c9037960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9035030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 --2- 172.21.15.50:0/3575085552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902a790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 -- 172.21.15.50:0/3575085552 >> 172.21.15.50:0/3575085552 conn(0x5567c9026920 msgr2=0x5567c9026d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 -- 172.21.15.50:0/3575085552 shutdown_connections 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7a440a2e90 0x7f7a440a5350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x5567c9027780 tx=0x7f7a50031510 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.805+0000 7f7a68df5740 1 -- 172.21.15.50:0/3575085552 wait complete. 2024-09-12T03:28:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 -- start start 2024-09-12T03:28:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 Processor -- start 2024-09-12T03:28:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 0x5567c9032530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9032a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9032fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567c9038050 con 0x5567c902acd0 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5567c8f19090 con 0x5567c902a3b0 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.806+0000 7f7a68df5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5567c90381d0 con 0x5567c9035570 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a68df5740 10 client.0 _ll_register_callbacks cb 0x5567c8f6df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 0x5567c9032530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 0x5567c9032530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:52720/0 (socket says 172.21.15.50:52720) 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2896795765 learned_addr learned my addr 172.21.15.50:0/2896795765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5e000640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9032a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5f400640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9032fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2896795765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 msgr2=0x5567c9032fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9032fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2896795765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 msgr2=0x5567c9032a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9032a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.807+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2896795765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c8d6f620 con 0x5567c902a3b0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.808+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 0x5567c9032530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005570 tx=0x7f7a50009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53a257d8400c573 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.808+0000 7f7a48c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.808+0000 7f7a48c00640 1 -- 172.21.15.50:0/2896795765 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50035040 con 0x5567c902a3b0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.808+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a4c005500 con 0x5567c902a3b0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.808+0000 7f7a48c00640 1 -- 172.21.15.50:0/2896795765 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7a500317b0 con 0x5567c902a3b0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.809+0000 7f7a48c00640 1 -- 172.21.15.50:0/2896795765 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a50004e70 con 0x5567c902a3b0 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.809+0000 7f7a68df5740 10 client.25313 fetch_fsmap learned FSMap version 26 2024-09-12T03:28:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.809+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567c9043f20 con 0x5567c902a3b0 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.809+0000 7f7a48c00640 1 -- 172.21.15.50:0/2896795765 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c005500 con 0x5567c902a3b0 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.809+0000 7f7a48c00640 1 -- 172.21.15.50:0/2896795765 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7a50044030 con 0x5567c902a3b0 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.809+0000 7f7a68df5740 10 client.25313 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.810+0000 7f7a68df5740 10 client.25313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.810+0000 7f7a68df5740 20 client.25313 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.810+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c9045400 0x5567c9047830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.810+0000 7f7a68df5740 4 client.25313 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.810+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c8f9d830 con 0x5567c9045400 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.810+0000 7f7a5e000640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c9045400 0x5567c9047830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.811+0000 7f7a5e000640 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c9045400 0x5567c9047830 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d2f61536eb3e87 server_cookie=d259e6ec4f2b2e1f in_seq=0 out_seq=0 2024-09-12T03:28:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.811+0000 7f7a48c00640 10 client.25313 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a48c00640 1 -- 172.21.15.50:0/2896795765 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567c8f9d830 con 0x5567c9045400 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a48c00640 10 client.25313 handle_command_reply: tid=0 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 client.25313 shutdown 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 msgr2=0x5567c9032530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 0x5567c9032530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005570 tx=0x7f7a50009260 comp rx=0 tx=0).stop 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 shutdown_connections 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c9045400 0x5567c9047830 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9032fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9032a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2896795765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902a3b0 0x5567c9032530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a48c00640 0 client.25313 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.812+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 >> 172.21.15.50:0/2896795765 conn(0x5567c9026920 msgr2=0x5567c902c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.813+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 shutdown_connections 2024-09-12T03:28:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.813+0000 7f7a68df5740 1 -- 172.21.15.50:0/2896795765 wait complete. 2024-09-12T03:28:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.813+0000 7f7a68df5740 20 client.25313 trim_cache size 0 max 16384 2024-09-12T03:28:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.842+0000 7f7a68df5740 1 Processor -- start 2024-09-12T03:28:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 -- start start 2024-09-12T03:28:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9034390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c904d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400127e0 con 0x5567c902acd0 2024-09-12T03:28:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400100b0 con 0x5567c9035570 2024-09-12T03:28:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a68df5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4000df00 con 0x5567c902a3b0 2024-09-12T03:28:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a5e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9034390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58224/0 (socket says 172.21.15.50:58224) 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.843+0000 7f7a5ea00640 1 -- 172.21.15.50:0/1304768190 learned_addr learned my addr 172.21.15.50:0/1304768190 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5f400640 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c904d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5ea00640 1 -- 172.21.15.50:0/1304768190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 msgr2=0x5567c904d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c904d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5ea00640 1 -- 172.21.15.50:0/1304768190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 msgr2=0x5567c9034390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9034390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5ea00640 1 -- 172.21.15.50:0/1304768190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c902a1e0 con 0x5567c902a3b0 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5f400640 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c904d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.844+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902ce70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a50003da0 tx=0x7f7a50005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8894257f4a052b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.845+0000 7f7a4b400640 1 -- 172.21.15.50:0/1304768190 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500035f0 con 0x5567c902a3b0 2024-09-12T03:28:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.845+0000 7f7a4b400640 1 -- 172.21.15.50:0/1304768190 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5004c070 con 0x5567c902a3b0 2024-09-12T03:28:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.845+0000 7f7a4b400640 1 -- 172.21.15.50:0/1304768190 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500315b0 con 0x5567c902a3b0 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 -- 172.21.15.50:0/1304768190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 msgr2=0x5567c902ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902ce70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a50003da0 tx=0x7f7a50005a40 comp rx=0 tx=0).stop 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 -- 172.21.15.50:0/1304768190 shutdown_connections 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c9035570 0x5567c904d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902acd0 0x5567c9034390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 --2- 172.21.15.50:0/1304768190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c902a3b0 0x5567c902ce70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 -- 172.21.15.50:0/1304768190 >> 172.21.15.50:0/1304768190 conn(0x5567c9026920 msgr2=0x5567c9033c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 -- 172.21.15.50:0/1304768190 shutdown_connections 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 -- 172.21.15.50:0/1304768190 wait complete. 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 -- start start 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.846+0000 7f7a68df5740 1 Processor -- start 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 0x5567c9039300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902acd0 0x5567c9039840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9039d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4000e570 con 0x5567c902a3b0 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4000f9e0 con 0x5567c902acd0 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4000ec40 con 0x5567c9035570 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a68df5740 10 client.0 _ll_register_callbacks cb 0x7f7a50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 0x5567c9039300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a5e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902acd0 0x5567c9039840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a5ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 0x5567c9039300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46176/0 (socket says 172.21.15.50:46176) 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.847+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2851781771 learned_addr learned my addr 172.21.15.50:0/2851781771 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5f400640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9039d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2851781771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 msgr2=0x5567c9039d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9039d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902acd0 msgr2=0x5567c9039840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902acd0 0x5567c9039840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5ea00640 1 -- 172.21.15.50:0/2851781771 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c90275b0 con 0x5567c902a3b0 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5e000640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902acd0 0x5567c9039840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a5ea00640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 0x5567c9039300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7a5002aa70 tx=0x7f7a50003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e80f871506ea25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.848+0000 7f7a48c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a48c00640 1 -- 172.21.15.50:0/2851781771 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500319c0 con 0x5567c902a3b0 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5004c070 con 0x5567c902a3b0 2024-09-12T03:28:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a48c00640 1 -- 172.21.15.50:0/2851781771 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7a50031b60 con 0x5567c902a3b0 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a48c00640 1 -- 172.21.15.50:0/2851781771 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a5006e050 con 0x5567c902a3b0 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a68df5740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567c90b0250 con 0x5567c902a3b0 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.849+0000 7f7a48c00640 1 -- 172.21.15.50:0/2851781771 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5004c070 con 0x5567c902a3b0 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a48c00640 1 -- 172.21.15.50:0/2851781771 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5005c020 con 0x5567c902a3b0 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a68df5740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a68df5740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a68df5740 20 client.15303 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c90b0900 0x5567c90b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a68df5740 4 client.15303 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:28:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.850+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567c8f97b80 con 0x5567c90b0900 2024-09-12T03:28:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.852+0000 7f7a5e000640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c90b0900 0x5567c90b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.852+0000 7f7a5e000640 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c90b0900 0x5567c90b2cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943f981be53aafb4 server_cookie=faf2ca79c7cec8d0 in_seq=0 out_seq=0 2024-09-12T03:28:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.852+0000 7f7a48c00640 10 client.15303 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a48c00640 1 -- 172.21.15.50:0/2851781771 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567c8f97b80 con 0x5567c90b0900 2024-09-12T03:28:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a48c00640 10 client.15303 handle_command_reply: tid=0 2024-09-12T03:28:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 client.15303 shutdown 2024-09-12T03:28:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 msgr2=0x5567c9039300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 0x5567c9039300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7a5002aa70 tx=0x7f7a50003fd0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 shutdown_connections 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5567c90b0900 0x5567c90b2cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567c9035570 0x5567c9039d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5567c902acd0 0x5567c9039840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 --2- 172.21.15.50:0/2851781771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567c902a3b0 0x5567c9039300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 >> 172.21.15.50:0/2851781771 conn(0x5567c9026920 msgr2=0x5567c90259c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a48c00640 0 client.15303 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 shutdown_connections 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.859+0000 7f7a68df5740 1 -- 172.21.15.50:0/2851781771 wait complete. 2024-09-12T03:28:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.860+0000 7f7a68df5740 20 client.15303 trim_cache size 0 max 16384 2024-09-12T03:28:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:28:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:28:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:28:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.860+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7a440a2e90 msgr2=0x7f7a440a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.860+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7a440a2e90 0x7f7a440a5350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x5567c9027780 tx=0x7f7a50031510 comp rx=0 tx=0).stop 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.860+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 msgr2=0x7f7a6019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.860+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 0x7f7a6019ed60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800eb10 tx=0x7f7a5800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 shutdown_connections 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a6010bf00 0x7f7a6019ed60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7a440a2e90 0x7f7a440a5350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a60103400 0x7f7a6019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 --2- 172.21.15.50:0/3089912945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a60102ae0 0x7f7a6019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 >> 172.21.15.50:0/3089912945 conn(0x7f7a600777c0 msgr2=0x7f7a60100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 shutdown_connections 2024-09-12T03:28:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:06.861+0000 7f7a5fe00640 1 -- 172.21.15.50:0/3089912945 wait complete. 2024-09-12T03:28:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:06 smithi050 ceph-mon[30378]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:28:07.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:06 smithi142 ceph-mon[35329]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:28:07.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:06 smithi082 ceph-mon[35046]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:28:09.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:08 smithi050 ceph-mon[30378]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:28:09.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:08 smithi142 ceph-mon[35329]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:28:09.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:08 smithi082 ceph-mon[35046]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:28:11.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:10 smithi050 ceph-mon[30378]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-12T03:28:11.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:10 smithi142 ceph-mon[35329]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-12T03:28:11.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:10 smithi082 ceph-mon[35046]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-12T03:28:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:12 smithi050 ceph-mon[30378]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:28:13.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:12 smithi142 ceph-mon[35329]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:28:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:12 smithi082 ceph-mon[35046]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:28:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:14 smithi050 ceph-mon[30378]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:28:15.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:14 smithi142 ceph-mon[35329]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:28:15.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:14 smithi082 ceph-mon[35046]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:28:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:16 smithi050 ceph-mon[30378]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:28:17.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:16 smithi142 ceph-mon[35329]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:28:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:16 smithi082 ceph-mon[35046]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:19.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:18 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:28:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:19.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:19.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:18 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:18 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:28:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 -- 172.21.15.50:0/3094714109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308076720 msgr2=0x7f3308076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 --2- 172.21.15.50:0/3094714109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308076720 0x7f3308076b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009860 tx=0x7f32fc02a340 comp rx=0 tx=0).stop 2024-09-12T03:28:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 -- 172.21.15.50:0/3094714109 shutdown_connections 2024-09-12T03:28:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 --2- 172.21.15.50:0/3094714109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3308112980 0x7f3308114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 --2- 172.21.15.50:0/3094714109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308077040 0x7f3308112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 --2- 172.21.15.50:0/3094714109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308076720 0x7f3308076b00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 -- 172.21.15.50:0/3094714109 >> 172.21.15.50:0/3094714109 conn(0x7f33080ff9e0 msgr2=0x7f3308101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 -- 172.21.15.50:0/3094714109 shutdown_connections 2024-09-12T03:28:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.978+0000 7f330fa00640 1 -- 172.21.15.50:0/3094714109 wait complete. 2024-09-12T03:28:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.979+0000 7f330fa00640 1 Processor -- start 2024-09-12T03:28:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.979+0000 7f330fa00640 1 -- start start 2024-09-12T03:28:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330fa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3308076720 0x7f33081a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330fa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 0x7f33081a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330fa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308112980 0x7f330819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330fa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33081184e0 con 0x7f3308076720 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330fa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3308118360 con 0x7f3308077040 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330fa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3308118660 con 0x7f3308112980 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3308076720 0x7f33081a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 0x7f33081a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 0x7f33081a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42120/0 (socket says 172.21.15.50:42120) 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 -- 172.21.15.50:0/736103762 learned_addr learned my addr 172.21.15.50:0/736103762 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f330da00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308112980 0x7f330819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 -- 172.21.15.50:0/736103762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308112980 msgr2=0x7f330819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308112980 0x7f330819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 -- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3308076720 msgr2=0x7f33081a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3308076720 0x7f33081a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.980+0000 7f3307e00640 1 -- 172.21.15.50:0/736103762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330819efc0 con 0x7f3308077040 2024-09-12T03:28:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f330da00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308112980 0x7f330819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f3307e00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 0x7f33081a4bc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32f400e9f0 tx=0x7f32f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f3305600640 1 -- 172.21.15.50:0/736103762 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f400cd80 con 0x7f3308077040 2024-09-12T03:28:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f3305600640 1 -- 172.21.15.50:0/736103762 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f4010910 con 0x7f3308077040 2024-09-12T03:28:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f3305600640 1 -- 172.21.15.50:0/736103762 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4018990 con 0x7f3308077040 2024-09-12T03:28:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330819f2b0 con 0x7f3308077040 2024-09-12T03:28:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.981+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081ab4d0 con 0x7f3308077040 2024-09-12T03:28:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.984+0000 7f3305600640 1 -- 172.21.15.50:0/736103762 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f32f4010ac0 con 0x7f3308077040 2024-09-12T03:28:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.984+0000 7f3312a59740 1 Processor -- start 2024-09-12T03:28:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 -- start start 2024-09-12T03:28:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de2d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558b9dd86100 con 0x558b9de20b00 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558b9dd2e1b0 con 0x558b9de201e0 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3312a59740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558b9dd820c0 con 0x558b9de2b3a0 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3307e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de2d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f330d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3307e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de2d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41072/0 (socket says 172.21.15.50:41072) 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f3307e00640 1 -- 172.21.15.50:0/3496014763 learned_addr learned my addr 172.21.15.50:0/3496014763 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.985+0000 7f330da00640 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3307e00640 1 -- 172.21.15.50:0/3496014763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 msgr2=0x558b9de205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3307e00640 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3307e00640 1 -- 172.21.15.50:0/3496014763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 msgr2=0x558b9de2ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3307e00640 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de2ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3307e00640 1 -- 172.21.15.50:0/3496014763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9da99210 con 0x558b9de2b3a0 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3305600640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f32f00a2ec0 0x7f32f00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f3307e00640 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de2d790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32f40cd040 tx=0x7f32f40d8570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72cbaadf6785d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.986+0000 7f330da00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f32f00a2ec0 0x7f32f00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f3305600640 1 -- 172.21.15.50:0/736103762 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f32f40cfec0 con 0x7f3308077040 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f32fa000640 1 -- 172.21.15.50:0/3496014763 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4098040 con 0x558b9de2b3a0 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f32fa000640 1 -- 172.21.15.50:0/3496014763 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f409c4a0 con 0x558b9de2b3a0 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f32fa000640 1 -- 172.21.15.50:0/3496014763 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f409b700 con 0x558b9de2b3a0 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f3312a59740 1 -- 172.21.15.50:0/3496014763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 msgr2=0x558b9de2d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f330da00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f32f00a2ec0 0x7f32f00a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f330806b1c0 tx=0x7f3300011040 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.987+0000 7f3312a59740 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de2d790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32f40cd040 tx=0x7f32f40d8570 comp rx=0 tx=0).stop 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 -- 172.21.15.50:0/3496014763 shutdown_connections 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de2d790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de2ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 --2- 172.21.15.50:0/3496014763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 -- 172.21.15.50:0/3496014763 >> 172.21.15.50:0/3496014763 conn(0x558b9de1c750 msgr2=0x558b9de1cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 -- 172.21.15.50:0/3496014763 shutdown_connections 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 -- 172.21.15.50:0/3496014763 wait complete. 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 -- start start 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.988+0000 7f3312a59740 1 Processor -- start 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de28300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de28840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de28d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558b9de2df70 con 0x558b9de201e0 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558b9dd0f090 con 0x558b9de2b3a0 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558b9de2e0f0 con 0x558b9de20b00 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3312a59740 10 client.0 _ll_register_callbacks cb 0x558b9dd63f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de28840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f330d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de28300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de28840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41080/0 (socket says 172.21.15.50:41080) 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 -- 172.21.15.50:0/2375053505 learned_addr learned my addr 172.21.15.50:0/2375053505 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f330da00640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de28d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 -- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 msgr2=0x558b9de28d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de28d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 -- 172.21.15.50:0/2375053505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 msgr2=0x558b9de28300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de28300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f3307e00640 1 -- 172.21.15.50:0/2375053505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9db65620 con 0x558b9de20b00 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.989+0000 7f330d000640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de28300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f3307e00640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de28840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32f409bd30 tx=0x7f32f4009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c452e3692dd020 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f32eea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f32eea00640 1 -- 172.21.15.50:0/2375053505 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f409b8e0 con 0x558b9de20b00 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f32eea00640 1 -- 172.21.15.50:0/2375053505 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f32f409c000 con 0x558b9de20b00 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f409c4a0 con 0x558b9de20b00 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f32eea00640 1 -- 172.21.15.50:0/2375053505 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32f409bb70 con 0x558b9de20b00 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f3312a59740 10 client.25324 fetch_fsmap learned FSMap version 26 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.990+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b9de2df70 con 0x558b9de20b00 2024-09-12T03:28:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f32eea00640 1 -- 172.21.15.50:0/2375053505 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f409c4a0 con 0x558b9de20b00 2024-09-12T03:28:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f32eea00640 1 -- 172.21.15.50:0/2375053505 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32f40a4580 con 0x558b9de20b00 2024-09-12T03:28:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f3312a59740 10 client.25324 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:28:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f3312a59740 10 client.25324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f3312a59740 20 client.25324 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f3312a59740 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9de3ad00 0x558b9de3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f3312a59740 4 client.25324 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b9dd93830 con 0x558b9de3ad00 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.991+0000 7f330d000640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9de3ad00 0x558b9de3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.992+0000 7f32eea00640 10 client.25324 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.992+0000 7f330d000640 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9de3ad00 0x558b9de3d0f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c95b0ce5cc02cc2 server_cookie=a69e01ec84ab4bf6 in_seq=0 out_seq=0 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f32eea00640 1 -- 172.21.15.50:0/2375053505 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b9dd93830 con 0x558b9de3ad00 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f32eea00640 10 client.25324 handle_command_reply: tid=0 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 client.25324 shutdown 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 msgr2=0x558b9de28840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de28840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32f409bd30 tx=0x7f32f4009700 comp rx=0 tx=0).stop 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 shutdown_connections 2024-09-12T03:28:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de28d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9de3ad00 0x558b9de3d0f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de28840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 --2- 172.21.15.50:0/2375053505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de28300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 >> 172.21.15.50:0/2375053505 conn(0x558b9de1c750 msgr2=0x558b9de224b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f32eea00640 0 client.25324 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.993+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 shutdown_connections 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.994+0000 7f3312a59740 1 -- 172.21.15.50:0/2375053505 wait complete. 2024-09-12T03:28:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:19.994+0000 7f3312a59740 20 client.25324 trim_cache size 0 max 16384 2024-09-12T03:28:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 Processor -- start 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 -- start start 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de42d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b7f80 con 0x558b9de201e0 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b5850 con 0x558b9de2b3a0 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3312a59740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b36a0 con 0x558b9de20b00 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f330d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f330da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de42d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3307e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f330da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de42d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42160/0 (socket says 172.21.15.50:42160) 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f330da00640 1 -- 172.21.15.50:0/1072464424 learned_addr learned my addr 172.21.15.50:0/1072464424 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.009+0000 7f3307e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41094/0 (socket says 172.21.15.50:41094) 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f330da00640 1 -- 172.21.15.50:0/1072464424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 msgr2=0x558b9de427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f330da00640 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f330da00640 1 -- 172.21.15.50:0/1072464424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 msgr2=0x558b9de223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f330da00640 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f330da00640 1 -- 172.21.15.50:0/1072464424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9de20930 con 0x558b9de2b3a0 2024-09-12T03:28:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f330da00640 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de42d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f330000b8c0 tx=0x7f3300017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d9574adc7f5944 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.010+0000 7f32f9600640 1 -- 172.21.15.50:0/1072464424 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330000bea0 con 0x558b9de2b3a0 2024-09-12T03:28:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f32f9600640 1 -- 172.21.15.50:0/1072464424 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f330002c070 con 0x558b9de2b3a0 2024-09-12T03:28:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f32f9600640 1 -- 172.21.15.50:0/1072464424 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300019620 con 0x558b9de2b3a0 2024-09-12T03:28:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 -- 172.21.15.50:0/1072464424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 msgr2=0x558b9de42d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de42d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f330000b8c0 tx=0x7f3300017000 comp rx=0 tx=0).stop 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 -- 172.21.15.50:0/1072464424 shutdown_connections 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de2b3a0 0x558b9de42d20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de20b00 0x558b9de427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 --2- 172.21.15.50:0/1072464424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de201e0 0x558b9de223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 -- 172.21.15.50:0/1072464424 >> 172.21.15.50:0/1072464424 conn(0x558b9de1c750 msgr2=0x558b9de21c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 -- 172.21.15.50:0/1072464424 shutdown_connections 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.011+0000 7f3312a59740 1 -- 172.21.15.50:0/1072464424 wait complete. 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 -- start start 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 Processor -- start 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de24dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de25300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b3d10 con 0x558b9de20b00 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b5180 con 0x558b9de201e0 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33081b43e0 con 0x558b9de2b3a0 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.012+0000 7f3312a59740 10 client.0 _ll_register_callbacks cb 0x7f3300009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f330d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de24dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f330d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de33840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42168/0 (socket says 172.21.15.50:42168) 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f330d000640 1 -- 172.21.15.50:0/3996905414 learned_addr learned my addr 172.21.15.50:0/3996905414 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f330da00640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de25300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 -- 172.21.15.50:0/3996905414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 msgr2=0x558b9de25300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de25300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 -- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 msgr2=0x558b9de33840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de33840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 -- 172.21.15.50:0/3996905414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9de258c0 con 0x558b9de20b00 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f330d000640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de33840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f3307e00640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de24dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x558b9de298b0 tx=0x7f32f40d8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b398528edc62f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.013+0000 7f32eea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f32eea00640 1 -- 172.21.15.50:0/3996905414 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f409b8e0 con 0x558b9de20b00 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f32eea00640 1 -- 172.21.15.50:0/3996905414 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f32f409c000 con 0x558b9de20b00 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f330002c070 con 0x558b9de20b00 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f32eea00640 1 -- 172.21.15.50:0/3996905414 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32f409b700 con 0x558b9de20b00 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-09-12T03:28:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b9dea5590 con 0x558b9de20b00 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f32eea00640 1 -- 172.21.15.50:0/3996905414 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f330002c070 con 0x558b9de20b00 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f32eea00640 1 -- 172.21.15.50:0/3996905414 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32f40a4580 con 0x558b9de20b00 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 20 client.15321 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9dea5a60 0x558b9dea7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 4 client.15321 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.014+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b9dd8db80 con 0x558b9dea5a60 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.015+0000 7f330d000640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9dea5a60 0x558b9dea7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.015+0000 7f330d000640 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9dea5a60 0x558b9dea7e50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f92399b73f326e server_cookie=b60e5331f4b96a63 in_seq=0 out_seq=0 2024-09-12T03:28:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.015+0000 7f32eea00640 10 client.15321 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f32eea00640 1 -- 172.21.15.50:0/3996905414 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b9dd8db80 con 0x558b9dea5a60 2024-09-12T03:28:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f32eea00640 10 client.15321 handle_command_reply: tid=0 2024-09-12T03:28:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 client.15321 shutdown 2024-09-12T03:28:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 msgr2=0x558b9de24dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de24dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x558b9de298b0 tx=0x7f32f40d8570 comp rx=0 tx=0).stop 2024-09-12T03:28:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 shutdown_connections 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558b9dea5a60 0x558b9dea7e50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b9de2b3a0 0x558b9de25300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b9de20b00 0x558b9de24dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 --2- 172.21.15.50:0/3996905414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b9de201e0 0x558b9de33840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 >> 172.21.15.50:0/3996905414 conn(0x558b9de1c750 msgr2=0x558b9de1b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f32eea00640 0 client.15321 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 shutdown_connections 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 1 -- 172.21.15.50:0/3996905414 wait complete. 2024-09-12T03:28:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.019+0000 7f3312a59740 20 client.15321 trim_cache size 0 max 16384 2024-09-12T03:28:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:28:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:28:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:28:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:28:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f32f00a2ec0 msgr2=0x7f32f00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f32f00a2ec0 0x7f32f00a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f330806b1c0 tx=0x7f3300011040 comp rx=0 tx=0).stop 2024-09-12T03:28:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 msgr2=0x7f33081a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 0x7f33081a4bc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32f400e9f0 tx=0x7f32f400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 shutdown_connections 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3308112980 0x7f330819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308077040 0x7f33081a4bc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f32f00a2ec0 0x7f32f00a5380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 --2- 172.21.15.50:0/736103762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3308076720 0x7f33081a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 >> 172.21.15.50:0/736103762 conn(0x7f33080ff9e0 msgr2=0x7f3308101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 shutdown_connections 2024-09-12T03:28:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:20.020+0000 7f330fa00640 1 -- 172.21.15.50:0/736103762 wait complete. 2024-09-12T03:28:20.168 INFO:tasks.workunit.client.0.smithi050.stderr:Updating files: 61% (7970/12903) Updating files: 62% (8000/12903) Updating files: 63% (8129/12903) Updating files: 64% (8258/12903) Updating files: 65% (8387/12903) Updating files: 66% (8516/12903) Updating files: 67% (8646/12903) Updating files: 68% (8775/12903) Updating files: 69% (8904/12903) Updating files: 70% (9033/12903) Updating files: 71% (9162/12903) Updating files: 72% (9291/12903) Updating files: 73% (9420/12903) Updating files: 74% (9549/12903) Updating files: 75% (9678/12903) Updating files: 76% (9807/12903) Updating files: 77% (9936/12903) Updating files: 78% (10065/12903) Updating files: 79% (10194/12903) Updating files: 80% (10323/12903) Updating files: 81% (10452/12903) Updating files: 82% (10581/12903) Updating files: 83% (10710/12903) Updating files: 84% (10839/12903) Updating files: 85% (10968/12903) Updating files: 86% (11097/12903) Updating files: 87% (11226/12903) Updating files: 88% (11355/12903) Updating files: 89% (11484/12903) Updating files: 90% (11613/12903) Updating files: 91% (11742/12903) Updating files: 92% (11871/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 97% (12528/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T03:28:20.876 INFO:tasks.workunit.client.0.smithi050.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr: git switch -c 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr:Or undo this operation with: 2024-09-12T03:28:20.877 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.878 INFO:tasks.workunit.client.0.smithi050.stderr: git switch - 2024-09-12T03:28:20.878 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.878 INFO:tasks.workunit.client.0.smithi050.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:28:20.878 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:20.878 INFO:tasks.workunit.client.0.smithi050.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T03:28:20.892 DEBUG:teuthology.orchestra.run.smithi050:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T03:28:20.983 INFO:tasks.workunit.client.0.smithi050.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T03:28:20.988 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:28:20.988 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T03:28:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:20 smithi050 ceph-mon[30378]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:28:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:20 smithi142 ceph-mon[35329]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:28:21.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:20 smithi082 ceph-mon[35046]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:28:21.729 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T03:28:21.819 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T03:28:21.886 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:28:21.890 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:28:21.890 INFO:tasks.workunit.client.0.smithi050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T03:28:21.955 INFO:tasks.workunit.client.0.smithi050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:28:21.962 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:28:21.962 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T03:28:21.999 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-09-12T03:28:22.000 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-09-12T03:28:22.000 DEBUG:teuthology.orchestra.run.smithi050:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-09-12T03:28:22.086 INFO:tasks.workunit.client.0.smithi050.stderr:+ set -e 2024-09-12T03:28:22.121 INFO:tasks.workunit.client.0.smithi050.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-09-12T03:28:22.121 INFO:tasks.workunit.client.0.smithi050.stderr:Cloning into 'xfstests-dev'... 2024-09-12T03:28:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:22 smithi050 ceph-mon[30378]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:22 smithi142 ceph-mon[35329]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:23.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:22 smithi082 ceph-mon[35046]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:24 smithi050 ceph-mon[30378]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-09-12T03:28:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:24 smithi142 ceph-mon[35329]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-09-12T03:28:25.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:24 smithi082 ceph-mon[35046]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-09-12T03:28:27.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:26 smithi142 ceph-mon[35329]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-12T03:28:27.361 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:26 smithi082 ceph-mon[35046]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-12T03:28:27.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:26 smithi050 ceph-mon[30378]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-12T03:28:29.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:28 smithi142 ceph-mon[35329]: pgmap v41: 129 pgs: 129 active+clean; 7.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 493 KiB/s wr, 7 op/s 2024-09-12T03:28:29.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:28 smithi082 ceph-mon[35046]: pgmap v41: 129 pgs: 129 active+clean; 7.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 493 KiB/s wr, 7 op/s 2024-09-12T03:28:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:28 smithi050 ceph-mon[30378]: pgmap v41: 129 pgs: 129 active+clean; 7.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 493 KiB/s wr, 7 op/s 2024-09-12T03:28:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:30 smithi142 ceph-mon[35329]: pgmap v42: 129 pgs: 129 active+clean; 16 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-09-12T03:28:31.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:30 smithi082 ceph-mon[35046]: pgmap v42: 129 pgs: 129 active+clean; 16 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-09-12T03:28:31.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:30 smithi050 ceph-mon[30378]: pgmap v42: 129 pgs: 129 active+clean; 16 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.147+0000 7f74dca00640 1 -- 172.21.15.50:0/2859395828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 msgr2=0x7f74d8103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.147+0000 7f74dca00640 1 --2- 172.21.15.50:0/2859395828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d8103f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009930 tx=0x7f74c002a420 comp rx=0 tx=0).stop 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 -- 172.21.15.50:0/2859395828 shutdown_connections 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 --2- 172.21.15.50:0/2859395828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d8104490 0x7f74d81109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 --2- 172.21.15.50:0/2859395828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d8103f50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 --2- 172.21.15.50:0/2859395828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b90 0x7f74d8109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 -- 172.21.15.50:0/2859395828 >> 172.21.15.50:0/2859395828 conn(0x7f74d80ff7d0 msgr2=0x7f74d8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 -- 172.21.15.50:0/2859395828 shutdown_connections 2024-09-12T03:28:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.148+0000 7f74dca00640 1 -- 172.21.15.50:0/2859395828 wait complete. 2024-09-12T03:28:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 Processor -- start 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 -- start start 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d8104490 0x7f74d819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b90 0x7f74d819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8113f90 con 0x7f74d8103af0 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8113e10 con 0x7f74d8109b90 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.149+0000 7f74dca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8114110 con 0x7f74d8104490 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52804/0 (socket says 172.21.15.50:52804) 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 -- 172.21.15.50:0/1213315334 learned_addr learned my addr 172.21.15.50:0/1213315334 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74cfe00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d8104490 0x7f74d819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d7200640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b90 0x7f74d819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 -- 172.21.15.50:0/1213315334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d8104490 msgr2=0x7f74d819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d8104490 0x7f74d819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 -- 172.21.15.50:0/1213315334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b90 msgr2=0x7f74d819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b90 0x7f74d819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 -- 172.21.15.50:0/1213315334 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d819f470 con 0x7f74d8103af0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.150+0000 7f74d6800640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d819a480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e9f0 tx=0x7f74c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.151+0000 7f74cf400640 1 -- 172.21.15.50:0/1213315334 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800cd80 con 0x7f74d8103af0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.151+0000 7f74cf400640 1 -- 172.21.15.50:0/1213315334 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8010910 con 0x7f74d8103af0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.151+0000 7f74cf400640 1 -- 172.21.15.50:0/1213315334 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8018990 con 0x7f74d8103af0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.151+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d819f760 con 0x7f74d8103af0 2024-09-12T03:28:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.151+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d81a6ff0 con 0x7f74d8103af0 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 Processor -- start 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74cf400640 1 -- 172.21.15.50:0/1213315334 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f74c8010ac0 con 0x7f74d8103af0 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 -- start start 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c8b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c8d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5609e4be6100 con 0x5609e4c8b570 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5609e4b8e1b0 con 0x5609e4c803b0 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.154+0000 7f74e046a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5609e4be20c0 con 0x5609e4c80cd0 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c8b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c8b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58236/0 (socket says 172.21.15.50:58236) 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 -- 172.21.15.50:0/1806849534 learned_addr learned my addr 172.21.15.50:0/1806849534 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74cfe00640 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c8d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 -- 172.21.15.50:0/1806849534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 msgr2=0x5609e4c80790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c80790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 -- 172.21.15.50:0/1806849534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 msgr2=0x5609e4c8d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c8d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 -- 172.21.15.50:0/1806849534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e48f9210 con 0x5609e4c80cd0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74cf400640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f74bc0a2e90 0x7f74bc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.155+0000 7f74d6800640 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c8b030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74c80cce50 tx=0x7f74c80d3f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad61d6132ed6dce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.156+0000 7f74d7200640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f74bc0a2e90 0x7f74bc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.156+0000 7f74cf400640 1 -- 172.21.15.50:0/1213315334 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f74c80cfcd0 con 0x7f74d8103af0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.156+0000 7f74c7400640 1 -- 172.21.15.50:0/1806849534 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8097f00 con 0x5609e4c80cd0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.156+0000 7f74c7400640 1 -- 172.21.15.50:0/1806849534 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c80dd070 con 0x5609e4c80cd0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.156+0000 7f74c7400640 1 -- 172.21.15.50:0/1806849534 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c809b5c0 con 0x5609e4c80cd0 2024-09-12T03:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 -- 172.21.15.50:0/1806849534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 msgr2=0x5609e4c8b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74d7200640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f74bc0a2e90 0x7f74bc0a5350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005fd0 tx=0x7f74d0009500 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c8b030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74c80cce50 tx=0x7f74c80d3f00 comp rx=0 tx=0).stop 2024-09-12T03:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 -- 172.21.15.50:0/1806849534 shutdown_connections 2024-09-12T03:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c8d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c8b030 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 --2- 172.21.15.50:0/1806849534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 -- 172.21.15.50:0/1806849534 >> 172.21.15.50:0/1806849534 conn(0x5609e4c7c920 msgr2=0x5609e4c7cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 -- 172.21.15.50:0/1806849534 shutdown_connections 2024-09-12T03:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 -- 172.21.15.50:0/1806849534 wait complete. 2024-09-12T03:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.157+0000 7f74e046a740 1 -- start start 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 Processor -- start 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c88690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5609e4c8dff0 con 0x5609e4c8b570 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5609e4b6f090 con 0x5609e4c803b0 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5609e4c8e170 con 0x5609e4c80cd0 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74e046a740 10 client.0 _ll_register_callbacks cb 0x5609e4bc3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74d6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c88690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74d6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c88690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:49618/0 (socket says 172.21.15.50:49618) 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74d7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74cfe00640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.158+0000 7f74d6800640 1 -- 172.21.15.50:0/2113890785 learned_addr learned my addr 172.21.15.50:0/2113890785 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74cfe00640 1 -- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 msgr2=0x5609e4c88690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74cfe00640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c88690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74cfe00640 1 -- 172.21.15.50:0/2113890785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 msgr2=0x5609e4c89110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74cfe00640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c89110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74cfe00640 1 -- 172.21.15.50:0/2113890785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e49c5620 con 0x5609e4c80cd0 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74d6800640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c88690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74cfe00640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c88bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f74c002a940 tx=0x7f74c00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d549da4d72122e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74bbe00640 1 -- 172.21.15.50:0/2113890785 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0035040 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c80dd070 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.159+0000 7f74bbe00640 1 -- 172.21.15.50:0/2113890785 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f74c0005450 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74bbe00640 1 -- 172.21.15.50:0/2113890785 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74c0005650 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 10 client.25342 fetch_fsmap learned FSMap version 26 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609e4c99e20 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74bbe00640 1 -- 172.21.15.50:0/2113890785 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c80dd070 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74bbe00640 1 -- 172.21.15.50:0/2113890785 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74c0003a30 con 0x5609e4c80cd0 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 10 client.25342 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 10 client.25342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 20 client.25342 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4c9b290 0x5609e4c9d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 4 client.25342 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.160+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609e4bf3830 con 0x5609e4c9b290 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.161+0000 7f74d6800640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4c9b290 0x5609e4c9d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.161+0000 7f74d6800640 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4c9b290 0x5609e4c9d680 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f46622e6ec299d server_cookie=f736cd7b01012645 in_seq=0 out_seq=0 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.161+0000 7f74bbe00640 10 client.25342 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74bbe00640 1 -- 172.21.15.50:0/2113890785 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609e4bf3830 con 0x5609e4c9b290 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74bbe00640 10 client.25342 handle_command_reply: tid=0 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 client.25342 shutdown 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 msgr2=0x5609e4c88bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c88bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f74c002a940 tx=0x7f74c00043b0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 shutdown_connections 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4c89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4c9b290 0x5609e4c9d680 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4c88bd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 --2- 172.21.15.50:0/2113890785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c88690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74bbe00640 0 client.25342 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.162+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 >> 172.21.15.50:0/2113890785 conn(0x5609e4c7c920 msgr2=0x5609e4c82630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.163+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 shutdown_connections 2024-09-12T03:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.163+0000 7f74e046a740 1 -- 172.21.15.50:0/2113890785 wait complete. 2024-09-12T03:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.163+0000 7f74e046a740 20 client.25342 trim_cache size 0 max 16384 2024-09-12T03:28:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 Processor -- start 2024-09-12T03:28:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 -- start start 2024-09-12T03:28:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4ca2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4ca3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81b39d0 con 0x5609e4c8b570 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81b12a0 con 0x5609e4c803b0 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.178+0000 7f74e046a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81af0f0 con 0x5609e4c80cd0 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4ca3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4ca3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52842/0 (socket says 172.21.15.50:52842) 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d7200640 1 -- 172.21.15.50:0/801823520 learned_addr learned my addr 172.21.15.50:0/801823520 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74cfe00640 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4ca2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 -- 172.21.15.50:0/801823520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 msgr2=0x5609e4ca2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4ca2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 -- 172.21.15.50:0/801823520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 msgr2=0x5609e4ca3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4ca3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 -- 172.21.15.50:0/801823520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e4c8b140 con 0x5609e4c803b0 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74d6800640 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8a4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74c8018460 tx=0x7f74c8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd5870c6fd51f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.179+0000 7f74c6a00640 1 -- 172.21.15.50:0/801823520 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c809b7a0 con 0x5609e4c803b0 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74c6a00640 1 -- 172.21.15.50:0/801823520 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c80d35b0 con 0x5609e4c803b0 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74c6a00640 1 -- 172.21.15.50:0/801823520 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c809ba30 con 0x5609e4c803b0 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 -- 172.21.15.50:0/801823520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 msgr2=0x5609e4c8a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8a4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74c8018460 tx=0x7f74c8009310 comp rx=0 tx=0).stop 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 -- 172.21.15.50:0/801823520 shutdown_connections 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c8b570 0x5609e4ca3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c80cd0 0x5609e4ca2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 --2- 172.21.15.50:0/801823520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8a4f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 -- 172.21.15.50:0/801823520 >> 172.21.15.50:0/801823520 conn(0x5609e4c7c920 msgr2=0x5609e4c89de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 -- 172.21.15.50:0/801823520 shutdown_connections 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.180+0000 7f74e046a740 1 -- 172.21.15.50:0/801823520 wait complete. 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 -- start start 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 Processor -- start 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 0x5609e4c8f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c8b570 0x5609e4d04f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81af760 con 0x5609e4c80cd0 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81b0bd0 con 0x5609e4c803b0 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.181+0000 7f74e046a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81afe30 con 0x5609e4c8b570 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74e046a740 10 client.0 _ll_register_callbacks cb 0x7f74c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 0x5609e4c8f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74d6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 0x5609e4c8f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52850/0 (socket says 172.21.15.50:52850) 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 -- 172.21.15.50:0/1679464511 learned_addr learned my addr 172.21.15.50:0/1679464511 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74d7200640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c8b570 0x5609e4d04f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 -- 172.21.15.50:0/1679464511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c8b570 msgr2=0x5609e4d04f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c8b570 0x5609e4d04f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 -- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 msgr2=0x5609e4c8f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 -- 172.21.15.50:0/1679464511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e4c90250 con 0x5609e4c80cd0 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74d6800640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74cfe00640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 0x5609e4c8f9a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74c0003dd0 tx=0x7f74c0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdd8985b569b1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.182+0000 7f74bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74bbe00640 1 -- 172.21.15.50:0/1679464511 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0058060 con 0x5609e4c80cd0 2024-09-12T03:28:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c80d35b0 con 0x5609e4c80cd0 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74bbe00640 1 -- 172.21.15.50:0/1679464511 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f74c0031bf0 con 0x5609e4c80cd0 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74bbe00640 1 -- 172.21.15.50:0/1679464511 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74c0035070 con 0x5609e4c80cd0 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74e046a740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609e4d06550 con 0x5609e4c80cd0 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.183+0000 7f74bbe00640 1 -- 172.21.15.50:0/1679464511 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c80d35b0 con 0x5609e4c80cd0 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74bbe00640 1 -- 172.21.15.50:0/1679464511 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74c0004990 con 0x5609e4c80cd0 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74e046a740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74e046a740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74e046a740 20 client.15339 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74e046a740 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4d06cd0 0x5609e4d090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74e046a740 4 client.15339 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:28:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609e4bedb80 con 0x5609e4d06cd0 2024-09-12T03:28:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74d6800640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4d06cd0 0x5609e4d090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74d6800640 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4d06cd0 0x5609e4d090f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1bd0fff3fd973ed server_cookie=712de039b101f975 in_seq=0 out_seq=0 2024-09-12T03:28:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.184+0000 7f74bbe00640 10 client.15339 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:32 smithi142 ceph-mon[35329]: pgmap v43: 129 pgs: 129 active+clean; 16 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-09-12T03:28:33.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:32 smithi082 ceph-mon[35046]: pgmap v43: 129 pgs: 129 active+clean; 16 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-09-12T03:28:33.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:32 smithi050 ceph-mon[30378]: pgmap v43: 129 pgs: 129 active+clean; 16 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74bbe00640 1 -- 172.21.15.50:0/1679464511 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609e4bedb80 con 0x5609e4d06cd0 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74bbe00640 10 client.15339 handle_command_reply: tid=0 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 client.15339 shutdown 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 msgr2=0x5609e4c8f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 0x5609e4c8f9a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74c0003dd0 tx=0x7f74c0005dc0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 shutdown_connections 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5609e4d06cd0 0x5609e4d090f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e4c8b570 0x5609e4d04f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609e4c80cd0 0x5609e4c8f9a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 --2- 172.21.15.50:0/1679464511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e4c803b0 0x5609e4c8f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74bbe00640 0 client.15339 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 >> 172.21.15.50:0/1679464511 conn(0x5609e4c7c920 msgr2=0x5609e4c88e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 shutdown_connections 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 1 -- 172.21.15.50:0/1679464511 wait complete. 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.983+0000 7f74e046a740 20 client.15339 trim_cache size 0 max 16384 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:28:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f74bc0a2e90 msgr2=0x7f74bc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f74bc0a2e90 0x7f74bc0a5350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005fd0 tx=0x7f74d0009500 comp rx=0 tx=0).stop 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 msgr2=0x7f74d819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d819a480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e9f0 tx=0x7f74c800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 shutdown_connections 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b90 0x7f74d819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74d8104490 0x7f74d819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f74bc0a2e90 0x7f74bc0a5350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 --2- 172.21.15.50:0/1213315334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f74d8103af0 0x7f74d819a480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 >> 172.21.15.50:0/1213315334 conn(0x7f74d80ff7d0 msgr2=0x7f74d80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 shutdown_connections 2024-09-12T03:28:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:33.984+0000 7f74dca00640 1 -- 172.21.15.50:0/1213315334 wait complete. 2024-09-12T03:28:35.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:34 smithi142 ceph-mon[35329]: pgmap v44: 129 pgs: 129 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T03:28:35.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:34 smithi082 ceph-mon[35046]: pgmap v44: 129 pgs: 129 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T03:28:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:34 smithi050 ceph-mon[30378]: pgmap v44: 129 pgs: 129 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T03:28:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:36 smithi142 ceph-mon[35329]: pgmap v45: 129 pgs: 129 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-09-12T03:28:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:36 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:37.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:36 smithi082 ceph-mon[35046]: pgmap v45: 129 pgs: 129 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-09-12T03:28:37.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:36 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:36 smithi050 ceph-mon[30378]: pgmap v45: 129 pgs: 129 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-09-12T03:28:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:36 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:37.783 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[65532]: *** Caught signal (Segmentation fault) ** 2024-09-12T03:28:37.783 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[65532]: in thread 7f15565ec640 thread_name:MR_Finisher 2024-09-12T03:28:37.783 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:37 smithi142 podman[69730]: 2024-09-12 03:28:37.463141536 +0000 UTC m=+0.020364988 container died 1c1bbd01c1d1f0fba25054bf3a697b6745ee7360f66dcebfa30a910d4c1c1829 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:28:38.501 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:38 smithi142 podman[69730]: 2024-09-12 03:28:38.243168295 +0000 UTC m=+0.800391741 container remove 1c1bbd01c1d1f0fba25054bf3a697b6745ee7360f66dcebfa30a910d4c1c1829 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:28:38.501 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:38 smithi142 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-09-12T03:28:38.793 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:38 smithi142 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-09-12T03:28:38.793 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:38 smithi142 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service: Consumed 1.412s CPU time. 2024-09-12T03:28:39.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:38 smithi142 ceph-mon[35329]: pgmap v46: 129 pgs: 129 active+clean; 21 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.7 MiB/s wr, 316 op/s 2024-09-12T03:28:39.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:38 smithi082 ceph-mon[35046]: pgmap v46: 129 pgs: 129 active+clean; 21 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.7 MiB/s wr, 316 op/s 2024-09-12T03:28:39.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:38 smithi050 ceph-mon[30378]: pgmap v46: 129 pgs: 129 active+clean; 21 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.7 MiB/s wr, 316 op/s 2024-09-12T03:28:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:40 smithi142 ceph-mon[35329]: pgmap v47: 129 pgs: 129 active+clean; 21 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-09-12T03:28:41.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:40 smithi082 ceph-mon[35046]: pgmap v47: 129 pgs: 129 active+clean; 21 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-09-12T03:28:41.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:40 smithi050 ceph-mon[30378]: pgmap v47: 129 pgs: 129 active+clean; 21 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-09-12T03:28:42.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:41 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:42.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:41 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:42.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:41 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:43.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:42 smithi142 ceph-mon[35329]: pgmap v48: 129 pgs: 129 active+clean; 21 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 574 KiB/s wr, 384 op/s 2024-09-12T03:28:43.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:42 smithi082 ceph-mon[35046]: pgmap v48: 129 pgs: 129 active+clean; 21 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 574 KiB/s wr, 384 op/s 2024-09-12T03:28:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:42 smithi050 ceph-mon[30378]: pgmap v48: 129 pgs: 129 active+clean; 21 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 574 KiB/s wr, 384 op/s 2024-09-12T03:28:43.931 INFO:tasks.workunit.client.0.smithi050.stderr:Updating files: 11% (494/4351) Updating files: 12% (523/4351) Updating files: 13% (566/4351) Updating files: 14% (610/4351) Updating files: 15% (653/4351) Updating files: 16% (697/4351) Updating files: 17% (740/4351) Updating files: 18% (784/4351) Updating files: 19% (827/4351) Updating files: 20% (871/4351) Updating files: 20% (882/4351) Updating files: 21% (914/4351) Updating files: 22% (958/4351) Updating files: 23% (1001/4351) Updating files: 24% (1045/4351) Updating files: 25% (1088/4351) Updating files: 26% (1132/4351) Updating files: 27% (1175/4351) Updating files: 28% (1219/4351) Updating files: 29% (1262/4351) Updating files: 29% (1303/4351) Updating files: 30% (1306/4351) Updating files: 31% (1349/4351) Updating files: 32% (1393/4351) Updating files: 33% (1436/4351) Updating files: 34% (1480/4351) Updating files: 35% (1523/4351) Updating files: 36% (1567/4351) Updating files: 37% (1610/4351) Updating files: 38% (1654/4351) Updating files: 39% (1697/4351) Updating files: 40% (1741/4351) Updating files: 41% (1784/4351) Updating files: 42% (1828/4351) Updating files: 43% (1871/4351) Updating files: 43% (1884/4351) Updating files: 44% (1915/4351) Updating files: 45% (1958/4351) Updating files: 46% (2002/4351) Updating files: 47% (2045/4351) Updating files: 48% (2089/4351) Updating files: 49% (2132/4351) Updating files: 50% (2176/4351) Updating files: 51% (2220/4351) Updating files: 52% (2263/4351) Updating files: 53% (2307/4351) Updating files: 54% (2350/4351) Updating files: 55% (2394/4351) Updating files: 55% (2426/4351) Updating files: 56% (2437/4351) Updating files: 57% (2481/4351) Updating files: 58% (2524/4351) Updating files: 59% (2568/4351) Updating files: 60% (2611/4351) Updating files: 61% (2655/4351) Updating files: 62% (2698/4351) Updating files: 63% (2742/4351) Updating files: 64% (2785/4351) Updating files: 64% (2809/4351) Updating files: 65% (2829/4351) Updating files: 66% (2872/4351) Updating files: 67% (2916/4351) Updating files: 68% (2959/4351) Updating files: 69% (3003/4351) Updating files: 69% (3040/4351) Updating files: 70% (3046/4351) Updating files: 71% (3090/4351) Updating files: 72% (3133/4351) Updating files: 73% (3177/4351) Updating files: 74% (3220/4351) Updating files: 75% (3264/4351) Updating files: 76% (3307/4351) Updating files: 77% (3351/4351) Updating files: 78% (3394/4351) Updating files: 79% (3438/4351) Updating files: 80% (3481/4351) Updating files: 81% (3525/4351) Updating files: 81% (3532/4351) Updating files: 82% (3568/4351) Updating files: 83% (3612/4351) Updating files: 84% (3655/4351) Updating files: 85% (3699/4351) Updating files: 86% (3742/4351) Updating files: 87% (3786/4351) Updating files: 88% (3829/4351) Updating files: 89% (3873/4351) Updating files: 90% (3916/4351) Updating files: 91% (3960/4351) Updating files: 92% (4003/4351) Updating files: 93% (4047/4351) Updating files: 93% (4089/4351) Updating files: 94% (4090/4351) Updating files: 95% (4134/4351) Updating files: 96% (4177/4351) Updating files: 96% (4184/4351) Updating files: 97% (4221/4351) Updating files: 98% (4264/4351) Updating files: 99% (4308/4351) Updating files: 100% (4351/4351) Updating files: 100% (4351/4351), done. 2024-09-12T03:28:43.943 INFO:tasks.workunit.client.0.smithi050.stderr:+ cd xfstests-dev 2024-09-12T03:28:43.944 INFO:tasks.workunit.client.0.smithi050.stderr:+ git checkout 0e5c12dfd008efc2848c98108c9237487e91ef35 2024-09-12T03:28:44.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:43 smithi142 ceph-mon[35329]: pgmap v49: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 792 KiB/s wr, 393 op/s 2024-09-12T03:28:44.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:43 smithi082 ceph-mon[35046]: pgmap v49: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 792 KiB/s wr, 393 op/s 2024-09-12T03:28:44.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:43 smithi050 ceph-mon[30378]: pgmap v49: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 792 KiB/s wr, 393 op/s 2024-09-12T03:28:45.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:44 smithi142 ceph-mon[35329]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-12T03:28:45.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:44 smithi082 ceph-mon[35046]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-12T03:28:45.365 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:44 smithi050 ceph-mon[30378]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-12T03:28:46.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:45 smithi142 ceph-mon[35329]: pgmap v50: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 743 KiB/s wr, 387 op/s 2024-09-12T03:28:46.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:45 smithi082 ceph-mon[35046]: pgmap v50: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 743 KiB/s wr, 387 op/s 2024-09-12T03:28:46.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:45 smithi050 ceph-mon[30378]: pgmap v50: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 743 KiB/s wr, 387 op/s 2024-09-12T03:28:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 -- 172.21.15.50:0/515640459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa574103f50 msgr2=0x7fa574104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 --2- 172.21.15.50:0/515640459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa574103f50 0x7fa574104330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa564009900 tx=0x7fa56402a400 comp rx=0 tx=0).stop 2024-09-12T03:28:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 -- 172.21.15.50:0/515640459 shutdown_connections 2024-09-12T03:28:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 --2- 172.21.15.50:0/515640459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5741090a0 0x7fa574110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 --2- 172.21.15.50:0/515640459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa574104900 0x7fa574108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 --2- 172.21.15.50:0/515640459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa574103f50 0x7fa574104330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.102+0000 7fa57ae00640 1 -- 172.21.15.50:0/515640459 >> 172.21.15.50:0/515640459 conn(0x7fa5740ff9e0 msgr2=0x7fa574101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.103+0000 7fa57ae00640 1 -- 172.21.15.50:0/515640459 shutdown_connections 2024-09-12T03:28:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.103+0000 7fa57ae00640 1 -- 172.21.15.50:0/515640459 wait complete. 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.103+0000 7fa57ae00640 1 Processor -- start 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 -- start start 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa574103f50 0x7fa57419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5741090a0 0x7fa57419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa57419ee60 0x7fa57419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa574114110 con 0x7fa574103f50 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa574113f90 con 0x7fa57419ee60 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa57ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa574114290 con 0x7fa5741090a0 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa573e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa574103f50 0x7fa57419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa573e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa574103f50 0x7fa57419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60756/0 (socket says 172.21.15.50:60756) 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa578e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa57419ee60 0x7fa57419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa573e00640 1 -- 172.21.15.50:0/819186006 learned_addr learned my addr 172.21.15.50:0/819186006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.104+0000 7fa573400640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5741090a0 0x7fa57419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa578e00640 1 -- 172.21.15.50:0/819186006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5741090a0 msgr2=0x7fa57419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa578e00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5741090a0 0x7fa57419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa578e00640 1 -- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa574103f50 msgr2=0x7fa57419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa578e00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa574103f50 0x7fa57419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa578e00640 1 -- 172.21.15.50:0/819186006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57419f950 con 0x7fa57419ee60 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa578e00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa57419ee60 0x7fa57419f310 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ec50 tx=0x7fa56c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa570c00640 1 -- 172.21.15.50:0/819186006 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c003b70 con 0x7fa57419ee60 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57419fc40 con 0x7fa57419ee60 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa570c00640 1 -- 172.21.15.50:0/819186006 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa56c003d10 con 0x7fa57419ee60 2024-09-12T03:28:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa570c00640 1 -- 172.21.15.50:0/819186006 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c010890 con 0x7fa57419ee60 2024-09-12T03:28:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.105+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5741a73e0 con 0x7fa57419ee60 2024-09-12T03:28:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa570c00640 1 -- 172.21.15.50:0/819186006 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa56c003fc0 con 0x7fa57419ee60 2024-09-12T03:28:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 Processor -- start 2024-09-12T03:28:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 -- start start 2024-09-12T03:28:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f9921e0 0x559e3f9925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 0x559e3f99ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e3f8f8100 con 0x559e3f992b00 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559e3f8a01b0 con 0x559e3f99d3a0 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.109+0000 7fa57dc58740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e3f8f40c0 con 0x559e3f9921e0 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 0x559e3f99ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 0x559e3f99ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60758/0 (socket says 172.21.15.50:60758) 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 -- 172.21.15.50:0/4126913592 learned_addr learned my addr 172.21.15.50:0/4126913592 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573400640 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f9921e0 0x559e3f9925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 -- 172.21.15.50:0/4126913592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f9921e0 msgr2=0x559e3f9925c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f9921e0 0x559e3f9925c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 -- 172.21.15.50:0/4126913592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 msgr2=0x559e3f99f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 -- 172.21.15.50:0/4126913592 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3f60b210 con 0x559e3f992b00 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa570c00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa5580a2da0 0x7fa5580a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573400640 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f9921e0 0x559e3f9925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa570c00640 1 -- 172.21.15.50:0/819186006 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa56c0cfd80 con 0x7fa57419ee60 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.110+0000 7fa573e00640 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 0x559e3f99ce60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa564005dd0 tx=0x7fa564005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dff3759251ae7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.111+0000 7fa573400640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa5580a2da0 0x7fa5580a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.111+0000 7fa569600640 1 -- 172.21.15.50:0/4126913592 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564040070 con 0x559e3f992b00 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.111+0000 7fa569600640 1 -- 172.21.15.50:0/4126913592 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa564004400 con 0x559e3f992b00 2024-09-12T03:28:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.111+0000 7fa569600640 1 -- 172.21.15.50:0/4126913592 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564035040 con 0x559e3f992b00 2024-09-12T03:28:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 -- 172.21.15.50:0/4126913592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 msgr2=0x559e3f99ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 0x559e3f99ce60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa564005dd0 tx=0x7fa564005c00 comp rx=0 tx=0).stop 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa573400640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa5580a2da0 0x7fa5580a5260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x559e3f98fad0 tx=0x7fa5600096c0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 -- 172.21.15.50:0/4126913592 shutdown_connections 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f992b00 0x559e3f99ce60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 --2- 172.21.15.50:0/4126913592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f9921e0 0x559e3f9925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 -- 172.21.15.50:0/4126913592 >> 172.21.15.50:0/4126913592 conn(0x559e3f98e750 msgr2=0x559e3f98eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 -- 172.21.15.50:0/4126913592 shutdown_connections 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 -- 172.21.15.50:0/4126913592 wait complete. 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 -- start start 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.112+0000 7fa57dc58740 1 Processor -- start 2024-09-12T03:28:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f9921e0 0x559e3f99a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f99a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e3f99ff70 con 0x559e3f9921e0 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559e3f881090 con 0x559e3f99d3a0 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e3f9a00f0 con 0x559e3f992b00 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa57dc58740 10 client.0 _ll_register_callbacks cb 0x559e3f8d5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa578e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa573e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f9921e0 0x559e3f99a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa573400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f99a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa578e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:49842/0 (socket says 172.21.15.50:49842) 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa578e00640 1 -- 172.21.15.50:0/2266379544 learned_addr learned my addr 172.21.15.50:0/2266379544 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa578e00640 1 -- 172.21.15.50:0/2266379544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 msgr2=0x559e3f99a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa578e00640 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f99a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.113+0000 7fa578e00640 1 -- 172.21.15.50:0/2266379544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f9921e0 msgr2=0x559e3f99a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa578e00640 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f9921e0 0x559e3f99a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa578e00640 1 -- 172.21.15.50:0/2266379544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3f6d7620 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa578e00640 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa56c098880 tx=0x7fa56c09bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc4778dd7f8ed58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa55e000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa55e000640 1 -- 172.21.15.50:0/2266379544 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c09c5b0 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa564004400 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa55e000640 1 -- 172.21.15.50:0/2266379544 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa56c09c110 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa55e000640 1 -- 172.21.15.50:0/2266379544 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa56c082050 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa57dc58740 10 client.25367 fetch_fsmap learned FSMap version 28 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e3f9abca0 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.114+0000 7fa55e000640 1 -- 172.21.15.50:0/2266379544 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa564004400 con 0x559e3f99d3a0 2024-09-12T03:28:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa55e000640 1 -- 172.21.15.50:0/2266379544 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa56c0a4720 con 0x559e3f99d3a0 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa57dc58740 10 client.25367 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa57dc58740 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa57dc58740 20 client.25367 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3f9ad160 0x559e3f9af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa57dc58740 4 client.25367 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3f905830 con 0x559e3f9ad160 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.115+0000 7fa573e00640 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3f9ad160 0x559e3f9af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.116+0000 7fa573e00640 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3f9ad160 0x559e3f9af5d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca740b15e77bfa server_cookie=e2f4336e90ec2460 in_seq=0 out_seq=0 2024-09-12T03:28:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.116+0000 7fa55e000640 10 client.25367 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.118+0000 7fa55e000640 1 -- 172.21.15.50:0/2266379544 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e3f905830 con 0x559e3f9ad160 2024-09-12T03:28:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.118+0000 7fa55e000640 10 client.25367 handle_command_reply: tid=0 2024-09-12T03:28:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.118+0000 7fa57dc58740 1 client.25367 shutdown 2024-09-12T03:28:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.118+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 msgr2=0x559e3f99adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.118+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa56c098880 tx=0x7fa56c09bd70 comp rx=0 tx=0).stop 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 shutdown_connections 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f99d3a0 0x559e3f99adf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3f9ad160 0x559e3f9af5d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f99a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2266379544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f9921e0 0x559e3f99a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa55e000640 0 client.25367 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 >> 172.21.15.50:0/2266379544 conn(0x559e3f98e750 msgr2=0x559e3f994460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 shutdown_connections 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 1 -- 172.21.15.50:0/2266379544 wait complete. 2024-09-12T03:28:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.119+0000 7fa57dc58740 20 client.25367 trim_cache size 0 max 16384 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 Processor -- start 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 -- start start 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741b3a70 con 0x559e3f99d3a0 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741b1340 con 0x559e3f9921e0 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.135+0000 7fa57dc58740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741af190 con 0x559e3f992b00 2024-09-12T03:28:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa578e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:49854/0 (socket says 172.21.15.50:49854) 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 -- 172.21.15.50:0/2204865604 learned_addr learned my addr 172.21.15.50:0/2204865604 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573400640 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 -- 172.21.15.50:0/2204865604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 msgr2=0x559e3f9b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 -- 172.21.15.50:0/2204865604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 msgr2=0x559e3f9975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 -- 172.21.15.50:0/2204865604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3f989d60 con 0x559e3f9921e0 2024-09-12T03:28:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573400640 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa573e00640 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9b4ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa564004cb0 tx=0x7fa564009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a467a6d4905d92b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa568c00640 1 -- 172.21.15.50:0/2204865604 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564040040 con 0x559e3f9921e0 2024-09-12T03:28:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa568c00640 1 -- 172.21.15.50:0/2204865604 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa564004400 con 0x559e3f9921e0 2024-09-12T03:28:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.136+0000 7fa568c00640 1 -- 172.21.15.50:0/2204865604 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564035040 con 0x559e3f9921e0 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 -- 172.21.15.50:0/2204865604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 msgr2=0x559e3f9b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9b4ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa564004cb0 tx=0x7fa564009260 comp rx=0 tx=0).stop 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 -- 172.21.15.50:0/2204865604 shutdown_connections 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2204865604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9b4ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 -- 172.21.15.50:0/2204865604 >> 172.21.15.50:0/2204865604 conn(0x559e3f98e750 msgr2=0x559e3f99c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 -- 172.21.15.50:0/2204865604 shutdown_connections 2024-09-12T03:28:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.137+0000 7fa57dc58740 1 -- 172.21.15.50:0/2204865604 wait complete. 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 -- start start 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 Processor -- start 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741af800 con 0x559e3f99d3a0 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741b0c70 con 0x559e3f9921e0 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.138+0000 7fa57dc58740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741afed0 con 0x559e3f992b00 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa57dc58740 10 client.0 _ll_register_callbacks cb 0x7fa56c003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:49862/0 (socket says 172.21.15.50:49862) 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 -- 172.21.15.50:0/2964647373 learned_addr learned my addr 172.21.15.50:0/2964647373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa578e00640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573400640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 -- 172.21.15.50:0/2964647373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 msgr2=0x559e3f9a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 -- 172.21.15.50:0/2964647373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 msgr2=0x559e3f9a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 -- 172.21.15.50:0/2964647373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3f98f3e0 con 0x559e3f9921e0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573400640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa573e00640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9a5340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa56402a440 tx=0x7fa564004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2700448e6ae7de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa55e000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.139+0000 7fa55e000640 1 -- 172.21.15.50:0/2964647373 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564035070 con 0x559e3f9921e0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa564004400 con 0x559e3f9921e0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa55e000640 1 -- 172.21.15.50:0/2964647373 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa5640050f0 con 0x559e3f9921e0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa55e000640 1 -- 172.21.15.50:0/2964647373 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa564040070 con 0x559e3f9921e0 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 10 client.25373 fetch_fsmap learned FSMap version 28 2024-09-12T03:28:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e3fa18490 con 0x559e3f9921e0 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa55e000640 1 -- 172.21.15.50:0/2964647373 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa564004400 con 0x559e3f9921e0 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa55e000640 1 -- 172.21.15.50:0/2964647373 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa564057070 con 0x559e3f9921e0 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 10 client.25373 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 20 client.25373 populate_metadata read hostname 'smithi050' 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3fa186c0 0x559e3fa1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 4 client.25373 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.140+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e3f8ffb80 con 0x559e3fa186c0 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.141+0000 7fa573400640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3fa186c0 0x559e3fa1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.141+0000 7fa573400640 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3fa186c0 0x559e3fa1aab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54893150a410a36 server_cookie=a6fa5f81e3419866 in_seq=0 out_seq=0 2024-09-12T03:28:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:47.141+0000 7fa55e000640 10 client.25373 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:48.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:48 smithi142 ceph-mon[35329]: pgmap v51: 129 pgs: 129 active+clean; 33 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.8 MiB/s wr, 572 op/s 2024-09-12T03:28:48.604 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:48 smithi050 ceph-mon[30378]: pgmap v51: 129 pgs: 129 active+clean; 33 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.8 MiB/s wr, 572 op/s 2024-09-12T03:28:48.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:48 smithi082 ceph-mon[35046]: pgmap v51: 129 pgs: 129 active+clean; 33 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.8 MiB/s wr, 572 op/s 2024-09-12T03:28:48.962 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:48 smithi142 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-09-12T03:28:48.963 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:48 smithi142 systemd[1]: Stopped Ceph mds.f for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:28:48.963 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:48 smithi142 systemd[1]: ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service: Consumed 1.412s CPU time. 2024-09-12T03:28:48.963 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:48 smithi142 systemd[1]: Starting Ceph mds.f for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:28:49.293 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:48 smithi142 podman[69833]: 2024-09-12 03:28:48.865772721 +0000 UTC m=+0.214197188 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:28:49.294 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:48 smithi142 podman[69833]: 2024-09-12 03:28:48.986142721 +0000 UTC m=+0.334567174 container create 58bf2d33b166973f068bc9e1dfab874142d4127fb4d52b54804ce8165dd60827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:28:49.793 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:49 smithi142 podman[69833]: 2024-09-12 03:28:49.306022167 +0000 UTC m=+0.654446634 container init 58bf2d33b166973f068bc9e1dfab874142d4127fb4d52b54804ce8165dd60827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T03:28:49.794 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:49 smithi142 podman[69833]: 2024-09-12 03:28:49.32326401 +0000 UTC m=+0.671688481 container start 58bf2d33b166973f068bc9e1dfab874142d4127fb4d52b54804ce8165dd60827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:28:49.794 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[69849]: starting mds.f at 2024-09-12T03:28:49.794 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:49 smithi142 bash[69833]: 58bf2d33b166973f068bc9e1dfab874142d4127fb4d52b54804ce8165dd60827 2024-09-12T03:28:49.794 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:28:49 smithi142 systemd[1]: Started Ceph mds.f for 389f22da-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:28:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.191+0000 7fa55e000640 1 -- 172.21.15.50:0/2964647373 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e3f8ffb80 con 0x559e3fa186c0 2024-09-12T03:28:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.191+0000 7fa55e000640 10 client.25373 handle_command_reply: tid=0 2024-09-12T03:28:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.191+0000 7fa57dc58740 1 client.25373 shutdown 2024-09-12T03:28:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 msgr2=0x559e3f9a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9a5340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa56402a440 tx=0x7fa564004aa0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 shutdown_connections 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e3f99d3a0 0x559e3f9a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x559e3fa186c0 0x559e3fa1aab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e3f992b00 0x559e3f9a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 --2- 172.21.15.50:0/2964647373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559e3f9921e0 0x559e3f9a5340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 >> 172.21.15.50:0/2964647373 conn(0x559e3f98e750 msgr2=0x559e3f993c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa55e000640 0 client.25373 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 shutdown_connections 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 1 -- 172.21.15.50:0/2964647373 wait complete. 2024-09-12T03:28:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57dc58740 20 client.25373 trim_cache size 0 max 16384 2024-09-12T03:28:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:28:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:28:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:28:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa5580a2da0 msgr2=0x7fa5580a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa5580a2da0 0x7fa5580a5260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x559e3f98fad0 tx=0x7fa5600096c0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa57419ee60 msgr2=0x7fa57419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa57419ee60 0x7fa57419f310 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ec50 tx=0x7fa56c00c6a0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 shutdown_connections 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa57419ee60 0x7fa57419f310 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5741090a0 0x7fa57419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa5580a2da0 0x7fa5580a5260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 --2- 172.21.15.50:0/819186006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa574103f50 0x7fa57419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 >> 172.21.15.50:0/819186006 conn(0x7fa5740ff9e0 msgr2=0x7fa574100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 shutdown_connections 2024-09-12T03:28:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:28:50.192+0000 7fa57ae00640 1 -- 172.21.15.50:0/819186006 wait complete. 2024-09-12T03:28:50.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:50 smithi050 ceph-mon[30378]: pgmap v52: 129 pgs: 129 active+clean; 38 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-09-12T03:28:50.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:50 smithi050 ceph-mon[30378]: daemon mds.f restarted 2024-09-12T03:28:50.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:50 smithi142 ceph-mon[35329]: pgmap v52: 129 pgs: 129 active+clean; 38 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-09-12T03:28:50.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:50 smithi142 ceph-mon[35329]: daemon mds.f restarted 2024-09-12T03:28:50.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:50 smithi082 ceph-mon[35046]: pgmap v52: 129 pgs: 129 active+clean; 38 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-09-12T03:28:50.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:50 smithi082 ceph-mon[35046]: daemon mds.f restarted 2024-09-12T03:28:51.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:51 smithi142 ceph-mon[35329]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:28:51.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6834/1308550298,v1:172.21.15.142:6835/1308550298] up:boot 2024-09-12T03:28:51.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:51 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T03:28:51.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:28:51.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:51 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:51.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:51 smithi082 ceph-mon[35046]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:28:51.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6834/1308550298,v1:172.21.15.142:6835/1308550298] up:boot 2024-09-12T03:28:51.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:51 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T03:28:51.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:28:51.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:51 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:51.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:51 smithi050 ceph-mon[30378]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:28:51.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6834/1308550298,v1:172.21.15.142:6835/1308550298] up:boot 2024-09-12T03:28:51.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:51 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T03:28:51.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:28:51.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:51 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:28:52.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:52 smithi142 ceph-mon[35329]: pgmap v54: 129 pgs: 129 active+clean; 38 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-12T03:28:52.603 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:52 smithi082 ceph-mon[35046]: pgmap v54: 129 pgs: 129 active+clean; 38 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-12T03:28:52.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:52 smithi050 ceph-mon[30378]: pgmap v54: 129 pgs: 129 active+clean; 38 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-12T03:28:54.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:54 smithi142 ceph-mon[35329]: pgmap v55: 129 pgs: 129 active+clean; 33 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-09-12T03:28:54.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:54 smithi082 ceph-mon[35046]: pgmap v55: 129 pgs: 129 active+clean; 33 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-09-12T03:28:54.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:54 smithi050 ceph-mon[30378]: pgmap v55: 129 pgs: 129 active+clean; 33 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-09-12T03:28:55.270 INFO:tasks.workunit.client.0.smithi050.stderr:Note: switching to '0e5c12dfd008efc2848c98108c9237487e91ef35'. 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:28:55.271 INFO:tasks.workunit.client.0.smithi050.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: git switch -c 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr:Or undo this operation with: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: git switch - 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr: 2024-09-12T03:28:55.272 INFO:tasks.workunit.client.0.smithi050.stderr:HEAD is now at 0e5c12df xfs/522: use reflink instead of crc as test feature 2024-09-12T03:28:55.281 INFO:tasks.workunit.client.0.smithi050.stderr:+ make -j4 2024-09-12T03:28:55.312 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize -cfi 2024-09-12T03:28:56.112 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-12T03:28:56.112 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file './config.guess' 2024-09-12T03:28:56.123 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file './config.sub' 2024-09-12T03:28:56.134 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file './install-sh' 2024-09-12T03:28:56.146 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file './ltmain.sh' 2024-09-12T03:28:56.205 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-12T03:28:56.206 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-12T03:28:56.218 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-12T03:28:56.232 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-12T03:28:56.244 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-12T03:28:56.258 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-12T03:28:56.274 INFO:tasks.workunit.client.0.smithi050.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-12T03:28:56.277 INFO:tasks.workunit.client.0.smithi050.stdout:cp include/install-sh . 2024-09-12T03:28:56.308 INFO:tasks.workunit.client.0.smithi050.stdout:aclocal -I m4 2024-09-12T03:28:56.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:56 smithi142 ceph-mon[35329]: pgmap v56: 129 pgs: 129 active+clean; 33 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-09-12T03:28:56.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:56 smithi082 ceph-mon[35046]: pgmap v56: 129 pgs: 129 active+clean; 33 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-09-12T03:28:56.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:56 smithi050 ceph-mon[30378]: pgmap v56: 129 pgs: 129 active+clean; 33 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 1.35k op/s 2024-09-12T03:28:57.880 INFO:tasks.workunit.client.0.smithi050.stdout:autoheader 2024-09-12T03:28:58.260 INFO:tasks.workunit.client.0.smithi050.stdout:autoconf 2024-09-12T03:28:58.493 INFO:tasks.workunit.client.0.smithi050.stdout:./configure \ 2024-09-12T03:28:58.493 INFO:tasks.workunit.client.0.smithi050.stdout: --libexecdir=/usr/lib \ 2024-09-12T03:28:58.493 INFO:tasks.workunit.client.0.smithi050.stdout: --exec_prefix=/var/lib 2024-09-12T03:28:58.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:28:58 smithi142 ceph-mon[35329]: pgmap v57: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 1.28k op/s 2024-09-12T03:28:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:28:58 smithi082 ceph-mon[35046]: pgmap v57: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 1.28k op/s 2024-09-12T03:28:58.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:28:58 smithi050 ceph-mon[30378]: pgmap v57: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 1.28k op/s 2024-09-12T03:28:58.766 INFO:tasks.workunit.client.0.smithi050.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-12T03:28:58.766 INFO:tasks.workunit.client.0.smithi050.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-12T03:28:58.769 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to print strings... printf 2024-09-12T03:28:58.769 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc... gcc 2024-09-12T03:28:58.906 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether the C compiler works... yes 2024-09-12T03:28:58.906 INFO:tasks.workunit.client.0.smithi050.stdout:checking for C compiler default output file name... a.out 2024-09-12T03:28:58.968 INFO:tasks.workunit.client.0.smithi050.stdout:checking for suffix of executables... 2024-09-12T03:28:59.031 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether we are cross compiling... no 2024-09-12T03:28:59.087 INFO:tasks.workunit.client.0.smithi050.stdout:checking for suffix of object files... o 2024-09-12T03:28:59.127 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T03:28:59.169 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether gcc accepts -g... yes 2024-09-12T03:28:59.255 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T03:28:59.262 INFO:tasks.workunit.client.0.smithi050.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-12T03:28:59.269 INFO:tasks.workunit.client.0.smithi050.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T03:28:59.272 INFO:tasks.workunit.client.0.smithi050.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T03:28:59.275 INFO:tasks.workunit.client.0.smithi050.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-12T03:28:59.283 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-12T03:28:59.287 INFO:tasks.workunit.client.0.smithi050.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-12T03:28:59.298 INFO:tasks.workunit.client.0.smithi050.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-12T03:29:00.270 INFO:tasks.workunit.client.0.smithi050.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-12T03:29:00.271 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether ln -s works... yes 2024-09-12T03:29:00.287 INFO:tasks.workunit.client.0.smithi050.stdout:checking the maximum length of command line arguments... 1572864 2024-09-12T03:29:00.294 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-12T03:29:00.294 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-12T03:29:00.294 INFO:tasks.workunit.client.0.smithi050.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-12T03:29:00.294 INFO:tasks.workunit.client.0.smithi050.stdout:checking for objdump... objdump 2024-09-12T03:29:00.294 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to recognize dependent libraries... pass_all 2024-09-12T03:29:00.295 INFO:tasks.workunit.client.0.smithi050.stdout:checking for dlltool... no 2024-09-12T03:29:00.295 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-12T03:29:00.296 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ar... ar 2024-09-12T03:29:00.404 INFO:tasks.workunit.client.0.smithi050.stdout:checking for archiver @FILE support... @ 2024-09-12T03:29:00.405 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strip... strip 2024-09-12T03:29:00.405 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ranlib... ranlib 2024-09-12T03:29:00.405 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gawk... gawk 2024-09-12T03:29:00.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:00 smithi142 ceph-mon[35329]: pgmap v58: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 1.26k op/s 2024-09-12T03:29:00.561 INFO:tasks.workunit.client.0.smithi050.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-12T03:29:00.564 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sysroot... no 2024-09-12T03:29:00.579 INFO:tasks.workunit.client.0.smithi050.stdout:checking for a working dd... /usr/bin/dd 2024-09-12T03:29:00.593 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-12T03:29:00.603 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:00 smithi082 ceph-mon[35046]: pgmap v58: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 1.26k op/s 2024-09-12T03:29:00.640 INFO:tasks.workunit.client.0.smithi050.stdout:checking for mt... no 2024-09-12T03:29:00.655 INFO:tasks.workunit.client.0.smithi050.stdout:checking if : is a manifest tool... no 2024-09-12T03:29:00.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:00 smithi050 ceph-mon[30378]: pgmap v58: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 1.26k op/s 2024-09-12T03:29:00.769 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T03:29:01.075 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ANSI C header files... yes 2024-09-12T03:29:01.133 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/types.h... yes 2024-09-12T03:29:01.198 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/stat.h... yes 2024-09-12T03:29:01.262 INFO:tasks.workunit.client.0.smithi050.stdout:checking for stdlib.h... yes 2024-09-12T03:29:01.329 INFO:tasks.workunit.client.0.smithi050.stdout:checking for string.h... yes 2024-09-12T03:29:01.396 INFO:tasks.workunit.client.0.smithi050.stdout:checking for memory.h... yes 2024-09-12T03:29:01.462 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strings.h... yes 2024-09-12T03:29:01.528 INFO:tasks.workunit.client.0.smithi050.stdout:checking for inttypes.h... yes 2024-09-12T03:29:01.594 INFO:tasks.workunit.client.0.smithi050.stdout:checking for stdint.h... yes 2024-09-12T03:29:01.663 INFO:tasks.workunit.client.0.smithi050.stdout:checking for unistd.h... yes 2024-09-12T03:29:01.734 INFO:tasks.workunit.client.0.smithi050.stdout:checking for dlfcn.h... yes 2024-09-12T03:29:01.745 INFO:tasks.workunit.client.0.smithi050.stdout:checking for objdir... .libs 2024-09-12T03:29:01.923 INFO:tasks.workunit.client.0.smithi050.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-12T03:29:01.923 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-12T03:29:01.980 INFO:tasks.workunit.client.0.smithi050.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-12T03:29:02.056 INFO:tasks.workunit.client.0.smithi050.stdout:checking if gcc static flag -static works... no 2024-09-12T03:29:02.141 INFO:tasks.workunit.client.0.smithi050.stdout:checking if gcc supports -c -o file.o... yes 2024-09-12T03:29:02.142 INFO:tasks.workunit.client.0.smithi050.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-12T03:29:02.156 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-12T03:29:02.239 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether -lc should be explicitly linked in... no 2024-09-12T03:29:02.345 INFO:tasks.workunit.client.0.smithi050.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-12T03:29:02.345 INFO:tasks.workunit.client.0.smithi050.stdout:checking how to hardcode library paths into programs... immediate 2024-09-12T03:29:02.348 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether stripping libraries is possible... yes 2024-09-12T03:29:02.348 INFO:tasks.workunit.client.0.smithi050.stdout:checking if libtool supports shared libraries... yes 2024-09-12T03:29:02.348 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether to build shared libraries... yes 2024-09-12T03:29:02.348 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether to build static libraries... yes 2024-09-12T03:29:02.362 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc... (cached) gcc 2024-09-12T03:29:02.419 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T03:29:02.420 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T03:29:02.420 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T03:29:02.420 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gmake... /usr/bin/gmake 2024-09-12T03:29:02.420 INFO:tasks.workunit.client.0.smithi050.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-09-12T03:29:02.421 INFO:tasks.workunit.client.0.smithi050.stdout:checking for tar... /bin/tar 2024-09-12T03:29:02.421 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gzip... /bin/gzip 2024-09-12T03:29:02.447 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether gcc -MM is supported... yes 2024-09-12T03:29:02.447 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sort... /bin/sort 2024-09-12T03:29:02.447 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether ln -s works... yes 2024-09-12T03:29:02.448 INFO:tasks.workunit.client.0.smithi050.stdout:checking for rpm... /bin/rpm 2024-09-12T03:29:02.454 INFO:tasks.workunit.client.0.smithi050.stdout:checking for rpmbuild... no 2024-09-12T03:29:02.534 INFO:tasks.workunit.client.0.smithi050.stdout:checking assert.h usability... yes 2024-09-12T03:29:02.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:02 smithi142 ceph-mon[35329]: pgmap v59: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-09-12T03:29:02.560 INFO:tasks.workunit.client.0.smithi050.stdout:checking assert.h presence... yes 2024-09-12T03:29:02.561 INFO:tasks.workunit.client.0.smithi050.stdout:checking for assert.h... yes 2024-09-12T03:29:02.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:02 smithi082 ceph-mon[35046]: pgmap v59: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-09-12T03:29:02.639 INFO:tasks.workunit.client.0.smithi050.stdout:checking bstring.h usability... no 2024-09-12T03:29:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:02 smithi050 ceph-mon[30378]: pgmap v59: 129 pgs: 129 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-09-12T03:29:02.680 INFO:tasks.workunit.client.0.smithi050.stdout:checking bstring.h presence... no 2024-09-12T03:29:02.680 INFO:tasks.workunit.client.0.smithi050.stdout:checking for bstring.h... no 2024-09-12T03:29:02.744 INFO:tasks.workunit.client.0.smithi050.stdout:checking libgen.h usability... yes 2024-09-12T03:29:02.770 INFO:tasks.workunit.client.0.smithi050.stdout:checking libgen.h presence... yes 2024-09-12T03:29:02.770 INFO:tasks.workunit.client.0.smithi050.stdout:checking for libgen.h... yes 2024-09-12T03:29:02.852 INFO:tasks.workunit.client.0.smithi050.stdout:checking dirent.h usability... yes 2024-09-12T03:29:02.880 INFO:tasks.workunit.client.0.smithi050.stdout:checking dirent.h presence... yes 2024-09-12T03:29:02.880 INFO:tasks.workunit.client.0.smithi050.stdout:checking for dirent.h... yes 2024-09-12T03:29:02.952 INFO:tasks.workunit.client.0.smithi050.stdout:checking errno.h usability... yes 2024-09-12T03:29:02.979 INFO:tasks.workunit.client.0.smithi050.stdout:checking errno.h presence... yes 2024-09-12T03:29:02.979 INFO:tasks.workunit.client.0.smithi050.stdout:checking for errno.h... yes 2024-09-12T03:29:03.061 INFO:tasks.workunit.client.0.smithi050.stdout:checking malloc.h usability... yes 2024-09-12T03:29:03.089 INFO:tasks.workunit.client.0.smithi050.stdout:checking malloc.h presence... yes 2024-09-12T03:29:03.090 INFO:tasks.workunit.client.0.smithi050.stdout:checking for malloc.h... yes 2024-09-12T03:29:03.168 INFO:tasks.workunit.client.0.smithi050.stdout:checking uuid.h usability... no 2024-09-12T03:29:03.210 INFO:tasks.workunit.client.0.smithi050.stdout:checking uuid.h presence... no 2024-09-12T03:29:03.211 INFO:tasks.workunit.client.0.smithi050.stdout:checking for uuid.h... no 2024-09-12T03:29:03.282 INFO:tasks.workunit.client.0.smithi050.stdout:checking uuid/uuid.h usability... yes 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 -- 172.21.15.50:0/2362299322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce4103400 msgr2=0x7efce410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 --2- 172.21.15.50:0/2362299322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce4103400 0x7efce410b8e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc02a480 comp rx=0 tx=0).stop 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 -- 172.21.15.50:0/2362299322 shutdown_connections 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 --2- 172.21.15.50:0/2362299322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce410bf00 0x7efce410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 --2- 172.21.15.50:0/2362299322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce4103400 0x7efce410b8e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 --2- 172.21.15.50:0/2362299322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce4102ae0 0x7efce4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 -- 172.21.15.50:0/2362299322 >> 172.21.15.50:0/2362299322 conn(0x7efce40777c0 msgr2=0x7efce40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 -- 172.21.15.50:0/2362299322 shutdown_connections 2024-09-12T03:29:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.310+0000 7efce3e00640 1 -- 172.21.15.50:0/2362299322 wait complete. 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 Processor -- start 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 -- start start 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce4102ae0 0x7efce4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce4103400 0x7efce4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 0x7efce419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efce4111d20 con 0x7efce4103400 2024-09-12T03:29:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efce4111ba0 con 0x7efce410bf00 2024-09-12T03:29:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efce4111ea0 con 0x7efce4102ae0 2024-09-12T03:29:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 0x7efce419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce4103400 0x7efce4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 0x7efce419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42348/0 (socket says 172.21.15.50:42348) 2024-09-12T03:29:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.311+0000 7efce3400640 1 -- 172.21.15.50:0/1851433222 learned_addr learned my addr 172.21.15.50:0/1851433222 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:03.314 INFO:tasks.workunit.client.0.smithi050.stdout:checking uuid/uuid.h presence... yes 2024-09-12T03:29:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce2a00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce4102ae0 0x7efce4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.315 INFO:tasks.workunit.client.0.smithi050.stdout:checking for uuid/uuid.h... yes 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3400640 1 -- 172.21.15.50:0/1851433222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce4102ae0 msgr2=0x7efce4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3400640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce4102ae0 0x7efce4198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3400640 1 -- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce4103400 msgr2=0x7efce4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3400640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce4103400 0x7efce4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3400640 1 -- 172.21.15.50:0/1851433222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce419d240 con 0x7efce410bf00 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3400640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 0x7efce419cb10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00eaa0 tx=0x7efcdc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efcdb400640 1 -- 172.21.15.50:0/1851433222 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cdb0 con 0x7efce410bf00 2024-09-12T03:29:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efcdb400640 1 -- 172.21.15.50:0/1851433222 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcdc005020 con 0x7efce410bf00 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efcdb400640 1 -- 172.21.15.50:0/1851433222 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0116b0 con 0x7efce410bf00 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.312+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce419d530 con 0x7efce410bf00 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.313+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce41a4dc0 con 0x7efce410bf00 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.315+0000 7efcdb400640 1 -- 172.21.15.50:0/1851433222 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efcdc005560 con 0x7efce410bf00 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.315+0000 7efcec7d9740 1 Processor -- start 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 -- start start 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11add6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 0x55f11ade1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11ade3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f11ad38100 con 0x55f11add67c0 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f11ace01b0 con 0x55f11ade1980 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcec7d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f11ad340c0 con 0x55f11add70e0 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11add6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efce2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 0x55f11ade1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efce2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 0x55f11ade1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41396/0 (socket says 172.21.15.50:41396) 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11add6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52778/0 (socket says 172.21.15.50:52778) 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efce2a00640 1 -- 172.21.15.50:0/4013249082 learned_addr learned my addr 172.21.15.50:0/4013249082 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.316+0000 7efcdbe00640 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11ade3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efce2a00640 1 -- 172.21.15.50:0/4013249082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 msgr2=0x55f11ade3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efce2a00640 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11ade3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efce2a00640 1 -- 172.21.15.50:0/4013249082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 msgr2=0x55f11add6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efce2a00640 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11add6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efce2a00640 1 -- 172.21.15.50:0/4013249082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11aa4b210 con 0x55f11add70e0 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efcdbe00640 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11ade3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efcdb400640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efcc80a2d70 0x7efcc80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efce2a00640 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 0x55f11ade1440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009cf0 tx=0x7efcd4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ab02d0efdfd6904 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efcdb400640 1 -- 172.21.15.50:0/1851433222 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7efcdc0cf8e0 con 0x7efce410bf00 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efcdbe00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efcc80a2d70 0x7efcc80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.317+0000 7efcd3e00640 1 -- 172.21.15.50:0/4013249082 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4004230 con 0x55f11add70e0 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcd3e00640 1 -- 172.21.15.50:0/4013249082 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd40043d0 con 0x55f11add70e0 2024-09-12T03:29:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcd3e00640 1 -- 172.21.15.50:0/4013249082 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd40198c0 con 0x55f11add70e0 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 -- 172.21.15.50:0/4013249082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 msgr2=0x55f11ade1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 0x55f11ade1440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009cf0 tx=0x7efcd4011810 comp rx=0 tx=0).stop 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcdbe00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efcc80a2d70 0x7efcc80a5230 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efccc000c00 tx=0x7efccc041040 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 -- 172.21.15.50:0/4013249082 shutdown_connections 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11ade3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add70e0 0x55f11ade1440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 --2- 172.21.15.50:0/4013249082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11add6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.318+0000 7efcec7d9740 1 -- 172.21.15.50:0/4013249082 >> 172.21.15.50:0/4013249082 conn(0x55f11add2d30 msgr2=0x55f11add3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.319+0000 7efcec7d9740 1 -- 172.21.15.50:0/4013249082 shutdown_connections 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.319+0000 7efcec7d9740 1 -- 172.21.15.50:0/4013249082 wait complete. 2024-09-12T03:29:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.319+0000 7efcec7d9740 1 -- start start 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.319+0000 7efcec7d9740 1 Processor -- start 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11adde8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11ade1980 0x55f11addf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f11ade4550 con 0x55f11add67c0 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f11acc1090 con 0x55f11add70e0 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f11ade46d0 con 0x55f11ade1980 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcec7d9740 10 client.0 _ll_register_callbacks cb 0x55f11ad15f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efce2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11adde8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42362/0 (socket says 172.21.15.50:42362) 2024-09-12T03:29:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 -- 172.21.15.50:0/2929799208 learned_addr learned my addr 172.21.15.50:0/2929799208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efce3400640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11ade1980 0x55f11addf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 -- 172.21.15.50:0/2929799208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11ade1980 msgr2=0x55f11addf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11ade1980 0x55f11addf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 -- 172.21.15.50:0/2929799208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 msgr2=0x55f11adde8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11adde8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efcdbe00640 1 -- 172.21.15.50:0/2929799208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11ab17620 con 0x55f11add70e0 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.320+0000 7efce2a00640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11adde8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcdbe00640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addedf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efccc033000 tx=0x7efccc005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4061c186a2f826 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcd0c00640 1 -- 172.21.15.50:0/2929799208 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc04e070 con 0x55f11add70e0 2024-09-12T03:29:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd40043d0 con 0x55f11add70e0 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcd0c00640 1 -- 172.21.15.50:0/2929799208 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7efccc0325c0 con 0x55f11add70e0 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcd0c00640 1 -- 172.21.15.50:0/2929799208 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efccc035040 con 0x55f11add70e0 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcec7d9740 10 client.25388 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f11adf0240 con 0x55f11add70e0 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.321+0000 7efcd0c00640 1 -- 172.21.15.50:0/2929799208 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd40043d0 con 0x55f11add70e0 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcd0c00640 1 -- 172.21.15.50:0/2929799208 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efccc003940 con 0x55f11add70e0 2024-09-12T03:29:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcec7d9740 10 client.25388 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcec7d9740 10 client.25388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcec7d9740 20 client.25388 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11adf16b0 0x55f11adf3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcec7d9740 4 client.25388 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11ad45830 con 0x55f11adf16b0 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.322+0000 7efce2a00640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11adf16b0 0x55f11adf3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.323+0000 7efce2a00640 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11adf16b0 0x55f11adf3b50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc95d1f5adecb1c server_cookie=d9028d380903e946 in_seq=0 out_seq=0 2024-09-12T03:29:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.323+0000 7efcd0c00640 10 client.25388 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcd0c00640 1 -- 172.21.15.50:0/2929799208 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f11ad45830 con 0x55f11adf16b0 2024-09-12T03:29:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcd0c00640 10 client.25388 handle_command_reply: tid=0 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 client.25388 shutdown 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 msgr2=0x55f11addedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addedf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efccc033000 tx=0x7efccc005b50 comp rx=0 tx=0).stop 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 shutdown_connections 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11adf16b0 0x55f11adf3b50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11ade1980 0x55f11addf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcd0c00640 0 client.25388 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addedf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2929799208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add67c0 0x55f11adde8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 >> 172.21.15.50:0/2929799208 conn(0x55f11add2d30 msgr2=0x55f11add8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 shutdown_connections 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.324+0000 7efcec7d9740 1 -- 172.21.15.50:0/2929799208 wait complete. 2024-09-12T03:29:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.325+0000 7efcec7d9740 20 client.25388 trim_cache size 0 max 16384 2024-09-12T03:29:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.339+0000 7efcec7d9740 1 Processor -- start 2024-09-12T03:29:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 -- start start 2024-09-12T03:29:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11ade0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add70e0 0x55f11adf9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11adf99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efce41af530 con 0x55f11add70e0 2024-09-12T03:29:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efce41acda0 con 0x55f11ade1980 2024-09-12T03:29:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcec7d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efce41aabf0 con 0x55f11add67c0 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efce3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11adf99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add70e0 0x55f11adf9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efce3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11adf99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42378/0 (socket says 172.21.15.50:42378) 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efce2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11ade0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.340+0000 7efce3400640 1 -- 172.21.15.50:0/2134515517 learned_addr learned my addr 172.21.15.50:0/2134515517 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce3400640 1 -- 172.21.15.50:0/2134515517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 msgr2=0x55f11ade0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce3400640 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11ade0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce3400640 1 -- 172.21.15.50:0/2134515517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add70e0 msgr2=0x55f11adf9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce3400640 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add70e0 0x55f11adf9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce3400640 1 -- 172.21.15.50:0/2134515517 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11add39c0 con 0x55f11ade1980 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce2a00640 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11ade0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efcdbe00640 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add70e0 0x55f11adf9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efce3400640 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11adf99a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00efd0 tx=0x7efcdc09fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc70fb6469507abf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efcd3400640 1 -- 172.21.15.50:0/2134515517 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc01f030 con 0x55f11ade1980 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efcd3400640 1 -- 172.21.15.50:0/2134515517 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcdc0051d0 con 0x55f11ade1980 2024-09-12T03:29:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.341+0000 7efcd3400640 1 -- 172.21.15.50:0/2134515517 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0a4490 con 0x55f11ade1980 2024-09-12T03:29:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 -- 172.21.15.50:0/2134515517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 msgr2=0x55f11adf99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11adf99a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00efd0 tx=0x7efcdc09fec0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 -- 172.21.15.50:0/2134515517 shutdown_connections 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11ade1980 0x55f11adf99a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11add70e0 0x55f11adf9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 --2- 172.21.15.50:0/2134515517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11ade0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 -- 172.21.15.50:0/2134515517 >> 172.21.15.50:0/2134515517 conn(0x55f11add2d30 msgr2=0x55f11ade0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 -- 172.21.15.50:0/2134515517 shutdown_connections 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 -- 172.21.15.50:0/2134515517 wait complete. 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.342+0000 7efcec7d9740 1 -- start start 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 Processor -- start 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11adea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11ade1980 0x55f11addbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efce41ab260 con 0x55f11ade1980 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efce41ac6d0 con 0x55f11add70e0 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efce41ab930 con 0x55f11add67c0 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcec7d9740 10 client.0 _ll_register_callbacks cb 0x7efccc005110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efce2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11adea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efce2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11adea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41420/0 (socket says 172.21.15.50:41420) 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efce2a00640 1 -- 172.21.15.50:0/1603648035 learned_addr learned my addr 172.21.15.50:0/1603648035 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.343+0000 7efce3400640 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11ade1980 0x55f11addbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcdbe00640 1 -- 172.21.15.50:0/1603648035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 msgr2=0x55f11adea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcdbe00640 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11adea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcdbe00640 1 -- 172.21.15.50:0/1603648035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11ade1980 msgr2=0x55f11addbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcdbe00640 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11ade1980 0x55f11addbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcdbe00640 1 -- 172.21.15.50:0/1603648035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11ae5b430 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcdbe00640 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addb780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efccc003d10 tx=0x7efccc0098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad729e93ca5abb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcd0c00640 1 -- 172.21.15.50:0/1603648035 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc04e050 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcdc0051d0 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.344+0000 7efcd0c00640 1 -- 172.21.15.50:0/1603648035 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7efccc0037f0 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcd0c00640 1 -- 172.21.15.50:0/1603648035 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efccc004f60 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 10 client.25394 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f11ae5c620 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcd0c00640 1 -- 172.21.15.50:0/1603648035 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcdc0051d0 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcd0c00640 1 -- 172.21.15.50:0/1603648035 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efccc0326e0 con 0x55f11add70e0 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 10 client.25394 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 10 client.25394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 20 client.25394 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11ae5ce50 0x55f11ae5f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 4 client.25394 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f11ad3fb80 con 0x55f11ae5ce50 2024-09-12T03:29:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.345+0000 7efce2a00640 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11ae5ce50 0x55f11ae5f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.346+0000 7efce2a00640 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11ae5ce50 0x55f11ae5f270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e8a8878d816a30 server_cookie=a4a3055002da2f31 in_seq=0 out_seq=0 2024-09-12T03:29:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:03.346+0000 7efcd0c00640 10 client.25394 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:03.601 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/uuid.h usability... no 2024-09-12T03:29:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcd0c00640 1 -- 172.21.15.50:0/1603648035 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f11ad3fb80 con 0x55f11ae5ce50 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcd0c00640 10 client.25394 handle_command_reply: tid=0 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 client.25394 shutdown 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 msgr2=0x55f11addb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addb780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efccc003d10 tx=0x7efccc0098b0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 shutdown_connections 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f11ade1980 0x55f11addbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11add70e0 0x55f11addb780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f11ae5ce50 0x55f11ae5f270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 --2- 172.21.15.50:0/1603648035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11add67c0 0x55f11adea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.196+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 >> 172.21.15.50:0/1603648035 conn(0x55f11add2d30 msgr2=0x55f11add8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.197+0000 7efcd0c00640 0 client.25394 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.197+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 shutdown_connections 2024-09-12T03:29:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.197+0000 7efcec7d9740 1 -- 172.21.15.50:0/1603648035 wait complete. 2024-09-12T03:29:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.197+0000 7efcec7d9740 20 client.25394 trim_cache size 0 max 16384 2024-09-12T03:29:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:29:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:29:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:29:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efcc80a2d70 msgr2=0x7efcc80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efcc80a2d70 0x7efcc80a5230 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efccc000c00 tx=0x7efccc041040 comp rx=0 tx=0).stop 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 msgr2=0x7efce419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 0x7efce419cb10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00eaa0 tx=0x7efcdc00ef60 comp rx=0 tx=0).stop 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 shutdown_connections 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce410bf00 0x7efce419cb10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efcc80a2d70 0x7efcc80a5230 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efce4103400 0x7efce4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 --2- 172.21.15.50:0/1851433222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce4102ae0 0x7efce4198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 >> 172.21.15.50:0/1851433222 conn(0x7efce40777c0 msgr2=0x7efce40fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 shutdown_connections 2024-09-12T03:29:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:04.198+0000 7efce3e00640 1 -- 172.21.15.50:0/1851433222 wait complete. 2024-09-12T03:29:04.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:04 smithi142 ceph-mon[35329]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-09-12T03:29:04.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:04 smithi082 ceph-mon[35046]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-09-12T03:29:04.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:04 smithi050 ceph-mon[30378]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-09-12T03:29:04.681 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/uuid.h presence... no 2024-09-12T03:29:04.681 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/uuid.h... no 2024-09-12T03:29:04.925 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/file.h usability... yes 2024-09-12T03:29:04.951 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/file.h presence... yes 2024-09-12T03:29:04.951 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/file.h... yes 2024-09-12T03:29:05.055 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fcntl.h usability... yes 2024-09-12T03:29:05.087 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fcntl.h presence... yes 2024-09-12T03:29:05.088 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/fcntl.h... yes 2024-09-12T03:29:05.173 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/syssgi.h usability... no 2024-09-12T03:29:05.214 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/syssgi.h presence... no 2024-09-12T03:29:05.214 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/syssgi.h... no 2024-09-12T03:29:05.334 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/param.h usability... yes 2024-09-12T03:29:05.377 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/param.h presence... yes 2024-09-12T03:29:05.378 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/param.h... yes 2024-09-12T03:29:05.399 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/stat.h... (cached) yes 2024-09-12T03:29:05.476 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/statvfs.h usability... yes 2024-09-12T03:29:05.501 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/statvfs.h presence... yes 2024-09-12T03:29:05.502 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/statvfs.h... yes 2024-09-12T03:29:05.574 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/time.h usability... yes 2024-09-12T03:29:05.601 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/time.h presence... yes 2024-09-12T03:29:05.601 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/time.h... yes 2024-09-12T03:29:05.677 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/ioctl.h usability... yes 2024-09-12T03:29:05.707 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/ioctl.h presence... yes 2024-09-12T03:29:05.707 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/ioctl.h... yes 2024-09-12T03:29:05.788 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/wait.h usability... yes 2024-09-12T03:29:05.816 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/wait.h presence... yes 2024-09-12T03:29:05.816 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/wait.h... yes 2024-09-12T03:29:05.838 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/types.h... (cached) yes 2024-09-12T03:29:05.851 INFO:tasks.workunit.client.0.smithi050.stdout:checking for strings.h... (cached) yes 2024-09-12T03:29:05.919 INFO:tasks.workunit.client.0.smithi050.stdout:checking err.h usability... yes 2024-09-12T03:29:05.949 INFO:tasks.workunit.client.0.smithi050.stdout:checking err.h presence... yes 2024-09-12T03:29:05.949 INFO:tasks.workunit.client.0.smithi050.stdout:checking for err.h... yes 2024-09-12T03:29:06.049 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/falloc.h usability... yes 2024-09-12T03:29:06.074 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/falloc.h presence... yes 2024-09-12T03:29:06.074 INFO:tasks.workunit.client.0.smithi050.stdout:checking for linux/falloc.h... yes 2024-09-12T03:29:06.443 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-09-12T03:29:06.486 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-09-12T03:29:06.486 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-12T03:29:06.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:06 smithi142 ceph-mon[35329]: pgmap v61: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 743 KiB/s wr, 174 op/s 2024-09-12T03:29:06.556 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fs/xfs_itable.h usability... no 2024-09-12T03:29:06.597 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/fs/xfs_itable.h presence... no 2024-09-12T03:29:06.597 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-12T03:29:06.603 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:06 smithi082 ceph-mon[35046]: pgmap v61: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 743 KiB/s wr, 174 op/s 2024-09-12T03:29:06.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:06 smithi050 ceph-mon[30378]: pgmap v61: 129 pgs: 129 active+clean; 39 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 743 KiB/s wr, 174 op/s 2024-09-12T03:29:06.665 INFO:tasks.workunit.client.0.smithi050.stdout:checking xfs/platform_defs.h usability... no 2024-09-12T03:29:06.709 INFO:tasks.workunit.client.0.smithi050.stdout:checking xfs/platform_defs.h presence... no 2024-09-12T03:29:06.709 INFO:tasks.workunit.client.0.smithi050.stdout:checking for xfs/platform_defs.h... no 2024-09-12T03:29:06.779 INFO:tasks.workunit.client.0.smithi050.stdout:checking btrfs/ioctl.h usability... no 2024-09-12T03:29:06.825 INFO:tasks.workunit.client.0.smithi050.stdout:checking btrfs/ioctl.h presence... no 2024-09-12T03:29:06.825 INFO:tasks.workunit.client.0.smithi050.stdout:checking for btrfs/ioctl.h... no 2024-09-12T03:29:06.894 INFO:tasks.workunit.client.0.smithi050.stdout:checking cifs/ioctl.h usability... no 2024-09-12T03:29:06.940 INFO:tasks.workunit.client.0.smithi050.stdout:checking cifs/ioctl.h presence... no 2024-09-12T03:29:06.940 INFO:tasks.workunit.client.0.smithi050.stdout:checking for cifs/ioctl.h... no 2024-09-12T03:29:07.021 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/mman.h usability... yes 2024-09-12T03:29:07.050 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/mman.h presence... yes 2024-09-12T03:29:07.050 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/mman.h... yes 2024-09-12T03:29:07.131 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/ext4.h usability... no 2024-09-12T03:29:07.175 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/ext4.h presence... no 2024-09-12T03:29:07.175 INFO:tasks.workunit.client.0.smithi050.stdout:checking for linux/ext4.h... no 2024-09-12T03:29:07.480 INFO:tasks.workunit.client.0.smithi050.stdout:checking for xfs/xfs_log_format.h... no 2024-09-12T03:29:07.569 INFO:tasks.workunit.client.0.smithi050.stdout:checking for uuid_compare... no 2024-09-12T03:29:07.637 INFO:tasks.workunit.client.0.smithi050.stdout:checking for uuid_compare in -luuid... yes 2024-09-12T03:29:07.720 INFO:tasks.workunit.client.0.smithi050.stdout:checking for xfs/xfs.h... yes 2024-09-12T03:29:07.772 INFO:tasks.workunit.client.0.smithi050.stdout:checking for xfs/libxfs.h... no 2024-09-12T03:29:07.822 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-12T03:29:07.886 INFO:tasks.workunit.client.0.smithi050.stdout:checking for xfs/xqm.h... yes 2024-09-12T03:29:07.977 INFO:tasks.workunit.client.0.smithi050.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-12T03:29:08.035 INFO:tasks.workunit.client.0.smithi050.stdout:checking xfs/handle.h usability... yes 2024-09-12T03:29:08.060 INFO:tasks.workunit.client.0.smithi050.stdout:checking xfs/handle.h presence... yes 2024-09-12T03:29:08.060 INFO:tasks.workunit.client.0.smithi050.stdout:checking for xfs/handle.h... yes 2024-09-12T03:29:08.140 INFO:tasks.workunit.client.0.smithi050.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-12T03:29:08.204 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/acl.h usability... yes 2024-09-12T03:29:08.233 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/acl.h presence... yes 2024-09-12T03:29:08.233 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/acl.h... yes 2024-09-12T03:29:08.294 INFO:tasks.workunit.client.0.smithi050.stdout:checking acl/libacl.h usability... yes 2024-09-12T03:29:08.322 INFO:tasks.workunit.client.0.smithi050.stdout:checking acl/libacl.h presence... yes 2024-09-12T03:29:08.323 INFO:tasks.workunit.client.0.smithi050.stdout:checking for acl/libacl.h... yes 2024-09-12T03:29:08.391 INFO:tasks.workunit.client.0.smithi050.stdout:checking for acl_init in -lacl... yes 2024-09-12T03:29:08.482 INFO:tasks.workunit.client.0.smithi050.stdout:checking gdbm-ndbm.h usability... no 2024-09-12T03:29:08.524 INFO:tasks.workunit.client.0.smithi050.stdout:checking gdbm-ndbm.h presence... no 2024-09-12T03:29:08.524 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gdbm-ndbm.h... no 2024-09-12T03:29:08.585 INFO:tasks.workunit.client.0.smithi050.stdout:checking gdbm/ndbm.h usability... no 2024-09-12T03:29:08.603 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:08 smithi082 ceph-mon[35046]: pgmap v62: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 547 op/s 2024-09-12T03:29:08.629 INFO:tasks.workunit.client.0.smithi050.stdout:checking gdbm/ndbm.h presence... no 2024-09-12T03:29:08.629 INFO:tasks.workunit.client.0.smithi050.stdout:checking for gdbm/ndbm.h... no 2024-09-12T03:29:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:08 smithi050 ceph-mon[30378]: pgmap v62: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 547 op/s 2024-09-12T03:29:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:08 smithi142 ceph-mon[35329]: pgmap v62: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 547 op/s 2024-09-12T03:29:08.892 INFO:tasks.workunit.client.0.smithi050.stdout:checking ndbm.h usability... no 2024-09-12T03:29:10.564 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:10 smithi050 ceph-mon[30378]: pgmap v63: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:29:10.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:10 smithi082 ceph-mon[35046]: pgmap v63: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:29:10.784 INFO:tasks.workunit.client.0.smithi050.stdout:checking ndbm.h presence... no 2024-09-12T03:29:10.784 INFO:tasks.workunit.client.0.smithi050.stdout:checking for ndbm.h... no 2024-09-12T03:29:10.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:10 smithi142 ceph-mon[35329]: pgmap v63: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:29:10.869 INFO:tasks.workunit.client.0.smithi050.stdout:checking libaio.h usability... yes 2024-09-12T03:29:10.922 INFO:tasks.workunit.client.0.smithi050.stdout:checking libaio.h presence... yes 2024-09-12T03:29:10.924 INFO:tasks.workunit.client.0.smithi050.stdout:checking for libaio.h... yes 2024-09-12T03:29:11.016 INFO:tasks.workunit.client.0.smithi050.stdout:checking liburing.h usability... no 2024-09-12T03:29:11.062 INFO:tasks.workunit.client.0.smithi050.stdout:checking liburing.h presence... no 2024-09-12T03:29:11.063 INFO:tasks.workunit.client.0.smithi050.stdout:checking for liburing.h... no 2024-09-12T03:29:11.124 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/capability.h usability... no 2024-09-12T03:29:11.169 INFO:tasks.workunit.client.0.smithi050.stdout:checking sys/capability.h presence... no 2024-09-12T03:29:11.169 INFO:tasks.workunit.client.0.smithi050.stdout:checking for sys/capability.h... no 2024-09-12T03:29:11.222 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/fiemap.h usability... yes 2024-09-12T03:29:11.247 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/fiemap.h presence... yes 2024-09-12T03:29:11.247 INFO:tasks.workunit.client.0.smithi050.stdout:checking for linux/fiemap.h... yes 2024-09-12T03:29:11.314 INFO:tasks.workunit.client.0.smithi050.stdout:checking for fallocate... yes 2024-09-12T03:29:11.394 INFO:tasks.workunit.client.0.smithi050.stdout:checking for open_by_handle_at... yes 2024-09-12T03:29:11.453 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/fs.h usability... yes 2024-09-12T03:29:11.479 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/fs.h presence... yes 2024-09-12T03:29:11.480 INFO:tasks.workunit.client.0.smithi050.stdout:checking for linux/fs.h... yes 2024-09-12T03:29:11.555 INFO:tasks.workunit.client.0.smithi050.stdout:checking btrfsutil.h usability... no 2024-09-12T03:29:11.597 INFO:tasks.workunit.client.0.smithi050.stdout:checking btrfsutil.h presence... no 2024-09-12T03:29:11.597 INFO:tasks.workunit.client.0.smithi050.stdout:checking for btrfsutil.h... no 2024-09-12T03:29:11.667 INFO:tasks.workunit.client.0.smithi050.stdout:checking for copy_file_range... yes 2024-09-12T03:29:11.741 INFO:tasks.workunit.client.0.smithi050.stdout:checking for SEEK_DATA... yes 2024-09-12T03:29:11.836 INFO:tasks.workunit.client.0.smithi050.stdout:checking for BMV_OF_SHARED... yes 2024-09-12T03:29:11.914 INFO:tasks.workunit.client.0.smithi050.stdout:checking for nftw... yes 2024-09-12T03:29:11.993 INFO:tasks.workunit.client.0.smithi050.stdout:checking for RLIMIT_NOFILE... yes 2024-09-12T03:29:12.069 INFO:tasks.workunit.client.0.smithi050.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-09-12T03:29:12.160 INFO:tasks.workunit.client.0.smithi050.stdout:checking for FICLONE... yes 2024-09-12T03:29:12.237 INFO:tasks.workunit.client.0.smithi050.stdout:checking for renameat2... yes 2024-09-12T03:29:12.513 INFO:tasks.workunit.client.0.smithi050.stdout:checking for reallocarray... yes 2024-09-12T03:29:12.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:12 smithi082 ceph-mon[35046]: pgmap v64: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:29:12.614 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct mount_attr... yes 2024-09-12T03:29:12.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:12 smithi050 ceph-mon[30378]: pgmap v64: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:29:12.714 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-12T03:29:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:12 smithi142 ceph-mon[35329]: pgmap v64: 129 pgs: 129 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:29:12.808 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-12T03:29:12.899 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-12T03:29:12.992 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-12T03:29:13.090 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-12T03:29:13.190 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-12T03:29:13.285 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-12T03:29:13.354 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/btrfs.h usability... yes 2024-09-12T03:29:13.380 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/btrfs.h presence... yes 2024-09-12T03:29:13.380 INFO:tasks.workunit.client.0.smithi050.stdout:checking for linux/btrfs.h... yes 2024-09-12T03:29:13.457 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/btrfs_tree.h usability... yes 2024-09-12T03:29:13.485 INFO:tasks.workunit.client.0.smithi050.stdout:checking linux/btrfs_tree.h presence... yes 2024-09-12T03:29:13.485 INFO:tasks.workunit.client.0.smithi050.stdout:checking for linux/btrfs_tree.h... yes 2024-09-12T03:29:13.547 INFO:tasks.workunit.client.0.smithi050.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-12T03:29:13.601 INFO:tasks.workunit.client.0.smithi050.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-12T03:29:13.633 INFO:tasks.workunit.client.0.smithi050.stdout:configure: creating ./config.status 2024-09-12T03:29:14.484 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: creating include/builddefs 2024-09-12T03:29:14.516 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: creating include/config.h 2024-09-12T03:29:14.541 INFO:tasks.workunit.client.0.smithi050.stdout:config.status: executing libtool commands 2024-09-12T03:29:14.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:14 smithi082 ceph-mon[35046]: pgmap v65: 129 pgs: 129 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 558 op/s 2024-09-12T03:29:14.654 INFO:tasks.workunit.client.0.smithi050.stdout:Building include 2024-09-12T03:29:14.654 INFO:tasks.workunit.client.0.smithi050.stdout:Building lib 2024-09-12T03:29:14.654 INFO:tasks.workunit.client.0.smithi050.stdout:Building tests 2024-09-12T03:29:14.663 INFO:tasks.workunit.client.0.smithi050.stdout: [LTDEP] 2024-09-12T03:29:14.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:14 smithi050 ceph-mon[30378]: pgmap v65: 129 pgs: 129 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 558 op/s 2024-09-12T03:29:14.685 INFO:tasks.workunit.client.0.smithi050.stdout:Building btrfs 2024-09-12T03:29:14.686 INFO:tasks.workunit.client.0.smithi050.stdout:Building ceph 2024-09-12T03:29:14.686 INFO:tasks.workunit.client.0.smithi050.stdout:Building cifs 2024-09-12T03:29:14.698 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-09-12T03:29:14.698 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-09-12T03:29:14.740 INFO:tasks.workunit.client.0.smithi050.stdout:Building ext4 2024-09-12T03:29:14.744 INFO:tasks.workunit.client.0.smithi050.stdout:Building f2fs 2024-09-12T03:29:14.756 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-09-12T03:29:14.764 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-09-12T03:29:14.771 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-09-12T03:29:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:14 smithi142 ceph-mon[35329]: pgmap v65: 129 pgs: 129 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 558 op/s 2024-09-12T03:29:14.814 INFO:tasks.workunit.client.0.smithi050.stdout:Building generic 2024-09-12T03:29:14.905 INFO:tasks.workunit.client.0.smithi050.stdout:Building nfs 2024-09-12T03:29:14.917 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-09-12T03:29:14.922 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-09-12T03:29:14.965 INFO:tasks.workunit.client.0.smithi050.stdout:Building ocfs2 2024-09-12T03:29:14.978 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-09-12T03:29:15.004 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dataascii.lo 2024-09-12T03:29:15.024 INFO:tasks.workunit.client.0.smithi050.stdout:Building overlay 2024-09-12T03:29:15.051 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-09-12T03:29:15.203 INFO:tasks.workunit.client.0.smithi050.stdout:Building perf 2024-09-12T03:29:15.215 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-09-12T03:29:15.253 INFO:tasks.workunit.client.0.smithi050.stdout:Building selftest 2024-09-12T03:29:15.266 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-09-12T03:29:15.268 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] databin.lo 2024-09-12T03:29:15.295 INFO:tasks.workunit.client.0.smithi050.stdout:Building shared 2024-09-12T03:29:15.305 INFO:tasks.workunit.client.0.smithi050.stdout:Building tmpfs 2024-09-12T03:29:15.309 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-09-12T03:29:15.312 INFO:tasks.workunit.client.0.smithi050.stdout:Building udf 2024-09-12T03:29:15.318 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-09-12T03:29:15.328 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-09-12T03:29:15.365 INFO:tasks.workunit.client.0.smithi050.stdout:Building xfs 2024-09-12T03:29:15.366 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] datapid.lo 2024-09-12T03:29:15.368 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] file_lock.lo 2024-09-12T03:29:15.489 INFO:tasks.workunit.client.0.smithi050.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-09-12T03:29:15.565 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] forker.lo 2024-09-12T03:29:15.588 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] pattern.lo 2024-09-12T03:29:15.646 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] open_flags.lo 2024-09-12T03:29:15.815 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] random_range.lo 2024-09-12T03:29:15.816 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] string_to_tokens.lo 2024-09-12T03:29:15.891 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] str_to_bytes.lo 2024-09-12T03:29:15.908 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] tlibio.lo 2024-09-12T03:29:16.037 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] write_log.lo 2024-09-12T03:29:16.162 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] random.lo 2024-09-12T03:29:16.402 INFO:tasks.workunit.client.0.smithi050.stdout: [LD] 2024-09-12T03:29:16.585 INFO:tasks.workunit.client.0.smithi050.stdout:Building ltp 2024-09-12T03:29:16.585 INFO:tasks.workunit.client.0.smithi050.stdout:Building src 2024-09-12T03:29:16.585 INFO:tasks.workunit.client.0.smithi050.stdout:Building m4 2024-09-12T03:29:16.585 INFO:tasks.workunit.client.0.smithi050.stdout:Building common 2024-09-12T03:29:16.593 INFO:tasks.workunit.client.0.smithi050.stdout: [DEP] 2024-09-12T03:29:16.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:16 smithi082 ceph-mon[35046]: pgmap v66: 129 pgs: 129 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 513 op/s 2024-09-12T03:29:16.605 INFO:tasks.workunit.client.0.smithi050.stdout: [DEP] 2024-09-12T03:29:16.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:16 smithi050 ceph-mon[30378]: pgmap v66: 129 pgs: 129 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 513 op/s 2024-09-12T03:29:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:16 smithi142 ceph-mon[35329]: pgmap v66: 129 pgs: 129 active+clean; 48 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 513 op/s 2024-09-12T03:29:16.898 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] doio 2024-09-12T03:29:16.898 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fsstress 2024-09-12T03:29:16.898 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fsx 2024-09-12T03:29:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 -- 172.21.15.50:0/2659224921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1008d0 msgr2=0x7feb0c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 --2- 172.21.15.50:0/2659224921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1008d0 0x7feb0c100cb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb00009930 tx=0x7feb0002a4b0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 -- 172.21.15.50:0/2659224921 shutdown_connections 2024-09-12T03:29:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 --2- 172.21.15.50:0/2659224921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb0c105cd0 0x7feb0c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 --2- 172.21.15.50:0/2659224921 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1011f0 0x7feb0c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 --2- 172.21.15.50:0/2659224921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1008d0 0x7feb0c100cb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 -- 172.21.15.50:0/2659224921 >> 172.21.15.50:0/2659224921 conn(0x7feb0c0777c0 msgr2=0x7feb0c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 -- 172.21.15.50:0/2659224921 shutdown_connections 2024-09-12T03:29:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.368+0000 7feb12400640 1 -- 172.21.15.50:0/2659224921 wait complete. 2024-09-12T03:29:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.370+0000 7feb12400640 1 Processor -- start 2024-09-12T03:29:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.370+0000 7feb12400640 1 -- start start 2024-09-12T03:29:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.370+0000 7feb12400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1008d0 0x7feb0c1a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb12400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1011f0 0x7feb0c1a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb12400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb0c105cd0 0x7feb0c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb12400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c118300 con 0x7feb0c1008d0 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb12400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c118180 con 0x7feb0c105cd0 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb12400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c118480 con 0x7feb0c1011f0 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb11000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1008d0 0x7feb0c1a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb0be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1011f0 0x7feb0c1a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb11000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1008d0 0x7feb0c1a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33638/0 (socket says 172.21.15.50:33638) 2024-09-12T03:29:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb11000640 1 -- 172.21.15.50:0/3624168250 learned_addr learned my addr 172.21.15.50:0/3624168250 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.371+0000 7feb11a00640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb0c105cd0 0x7feb0c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb0be00640 1 -- 172.21.15.50:0/3624168250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb0c105cd0 msgr2=0x7feb0c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb0be00640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb0c105cd0 0x7feb0c19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb0be00640 1 -- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1008d0 msgr2=0x7feb0c1a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb0be00640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1008d0 0x7feb0c1a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb0be00640 1 -- 172.21.15.50:0/3624168250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c19ef30 con 0x7feb0c1011f0 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb0be00640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1011f0 0x7feb0c1a4af0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9f0 tx=0x7feaf800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb09600640 1 -- 172.21.15.50:0/3624168250 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf800ccc0 con 0x7feb0c1011f0 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb09600640 1 -- 172.21.15.50:0/3624168250 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf8010910 con 0x7feb0c1011f0 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.372+0000 7feb09600640 1 -- 172.21.15.50:0/3624168250 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8018960 con 0x7feb0c1011f0 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.373+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c19f220 con 0x7feb0c1011f0 2024-09-12T03:29:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.373+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1ab530 con 0x7feb0c1011f0 2024-09-12T03:29:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 Processor -- start 2024-09-12T03:29:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 -- start start 2024-09-12T03:29:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f463b0 0x564178f46790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f51570 0x564178f53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564178eac100 con 0x564178f463b0 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564178e541b0 con 0x564178f51570 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.378+0000 7feb16883740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564178ea80c0 con 0x564178f46cd0 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.379+0000 7feb11000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.379+0000 7feb11000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f51030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56372/0 (socket says 172.21.15.50:56372) 2024-09-12T03:29:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.379+0000 7feb11000640 1 -- 172.21.15.50:0/2114453695 learned_addr learned my addr 172.21.15.50:0/2114453695 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.379+0000 7feb11a00640 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f463b0 0x564178f46790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb11000640 1 -- 172.21.15.50:0/2114453695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f51570 msgr2=0x564178f53960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb11000640 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f51570 0x564178f53960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb11000640 1 -- 172.21.15.50:0/2114453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f463b0 msgr2=0x564178f46790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb11000640 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f463b0 0x564178f46790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb09600640 1 -- 172.21.15.50:0/3624168250 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feaf8010ac0 con 0x7feb0c1011f0 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb11000640 1 -- 172.21.15.50:0/2114453695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564178bbf210 con 0x564178f46cd0 2024-09-12T03:29:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.380+0000 7feb11000640 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f51030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb00009850 tx=0x7feb000314d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2187a3a41a2ac68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.381+0000 7feafe000640 1 -- 172.21.15.50:0/2114453695 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00040070 con 0x564178f46cd0 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.381+0000 7feafe000640 1 -- 172.21.15.50:0/2114453695 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb000043f0 con 0x564178f46cd0 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.381+0000 7feafe000640 1 -- 172.21.15.50:0/2114453695 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00035040 con 0x564178f46cd0 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 -- 172.21.15.50:0/2114453695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 msgr2=0x564178f51030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f51030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb00009850 tx=0x7feb000314d0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 -- 172.21.15.50:0/2114453695 shutdown_connections 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f51570 0x564178f53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f51030 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 --2- 172.21.15.50:0/2114453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f463b0 0x564178f46790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 -- 172.21.15.50:0/2114453695 >> 172.21.15.50:0/2114453695 conn(0x564178f42920 msgr2=0x564178f42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 -- 172.21.15.50:0/2114453695 shutdown_connections 2024-09-12T03:29:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.382+0000 7feb16883740 1 -- 172.21.15.50:0/2114453695 wait complete. 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.383+0000 7feb16883740 1 -- start start 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.383+0000 7feb09600640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7feaf40a2cd0 0x7feaf40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 Processor -- start 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 0x564178f4e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f4f0a0 0x564178f5d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564178f540b0 con 0x564178f46cd0 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564178e35090 con 0x564178f4f0a0 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564178f54230 con 0x564178f463b0 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb16883740 10 client.0 _ll_register_callbacks cb 0x564178e89f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.384+0000 7feb11a00640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7feaf40a2cd0 0x7feaf40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 0x564178f4e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 0x564178f4e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56378/0 (socket says 172.21.15.50:56378) 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11000640 1 -- 172.21.15.50:0/1977618332 learned_addr learned my addr 172.21.15.50:0/1977618332 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11a00640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f4f0a0 0x564178f5d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11000640 1 -- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f4f0a0 msgr2=0x564178f5d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11000640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f4f0a0 0x564178f5d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb11000640 1 -- 172.21.15.50:0/1977618332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 msgr2=0x564178f4eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.385+0000 7feb09600640 1 -- 172.21.15.50:0/3624168250 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7feaf80cfc60 con 0x7feb0c1011f0 2024-09-12T03:29:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.386+0000 7feb11a00640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7feaf40a2cd0 0x7feaf40a5190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb04005fd0 tx=0x7feb04003e50 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.386+0000 7feb0be00640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.386+0000 7feb11000640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.386+0000 7feb11000640 1 -- 172.21.15.50:0/1977618332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564178c8b620 con 0x564178f463b0 2024-09-12T03:29:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.386+0000 7feb0be00640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.387+0000 7feb11000640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 0x564178f4e620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb0002aa20 tx=0x7feb00009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b72f94224f8917 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.387+0000 7feaf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.388+0000 7feaf2a00640 1 -- 172.21.15.50:0/1977618332 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0004d020 con 0x564178f463b0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.388+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb000043f0 con 0x564178f463b0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.388+0000 7feaf2a00640 1 -- 172.21.15.50:0/1977618332 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7feb00005250 con 0x564178f463b0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.388+0000 7feaf2a00640 1 -- 172.21.15.50:0/1977618332 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb00058050 con 0x564178f463b0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.388+0000 7feb16883740 10 client.25396 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.388+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564178f540b0 con 0x564178f463b0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.389+0000 7feaf2a00640 1 -- 172.21.15.50:0/1977618332 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb000043f0 con 0x564178f463b0 2024-09-12T03:29:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feaf2a00640 1 -- 172.21.15.50:0/1977618332 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb00031c90 con 0x564178f463b0 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb16883740 10 client.25396 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb16883740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb16883740 20 client.25396 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb16883740 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178f61520 0x564178f63910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb16883740 4 client.25396 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564178eb9830 con 0x564178f61520 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.390+0000 7feb0be00640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178f61520 0x564178f63910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.391+0000 7feb0be00640 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178f61520 0x564178f63910 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708bfca22abfa884 server_cookie=b0e3e1333e1f3606 in_seq=0 out_seq=0 2024-09-12T03:29:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.391+0000 7feaf2a00640 10 client.25396 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feaf2a00640 1 -- 172.21.15.50:0/1977618332 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564178eb9830 con 0x564178f61520 2024-09-12T03:29:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feaf2a00640 10 client.25396 handle_command_reply: tid=0 2024-09-12T03:29:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 client.25396 shutdown 2024-09-12T03:29:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 msgr2=0x564178f4e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 0x564178f4e620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb0002aa20 tx=0x7feb00009260 comp rx=0 tx=0).stop 2024-09-12T03:29:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 shutdown_connections 2024-09-12T03:29:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f4f0a0 0x564178f5d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178f61520 0x564178f63910 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 --2- 172.21.15.50:0/1977618332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f463b0 0x564178f4e620 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 >> 172.21.15.50:0/1977618332 conn(0x564178f42920 msgr2=0x564178f48680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.392+0000 7feaf2a00640 0 client.25396 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.393+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 shutdown_connections 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.393+0000 7feb16883740 1 -- 172.21.15.50:0/1977618332 wait complete. 2024-09-12T03:29:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.393+0000 7feb16883740 20 client.25396 trim_cache size 0 max 16384 2024-09-12T03:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 Processor -- start 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 -- start start 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f5d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f4f0a0 0x564178f4d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b7d80 con 0x564178f46cd0 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b55f0 con 0x564178f463b0 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.426+0000 7feb16883740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b3440 con 0x564178f4f0a0 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f5d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f5d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:52854/0 (socket says 172.21.15.50:52854) 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 -- 172.21.15.50:0/2997956452 learned_addr learned my addr 172.21.15.50:0/2997956452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb0be00640 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11a00640 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f4f0a0 0x564178f4d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 -- 172.21.15.50:0/2997956452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f4f0a0 msgr2=0x564178f4d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f4f0a0 0x564178f4d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 -- 172.21.15.50:0/2997956452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 msgr2=0x564178f4ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb11000640 1 -- 172.21.15.50:0/2997956452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564178f4e3f0 con 0x564178f463b0 2024-09-12T03:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.427+0000 7feb0be00640 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.428+0000 7feb11000640 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f5d430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb00003670 tx=0x7feb00005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa02dbfa9baaf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.428+0000 7feafd600640 1 -- 172.21.15.50:0/2997956452 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0004d040 con 0x564178f463b0 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.428+0000 7feafd600640 1 -- 172.21.15.50:0/2997956452 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb000046c0 con 0x564178f463b0 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.428+0000 7feafd600640 1 -- 172.21.15.50:0/2997956452 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00040070 con 0x564178f463b0 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 -- 172.21.15.50:0/2997956452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 msgr2=0x564178f5d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f5d430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feb00003670 tx=0x7feb00005df0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 -- 172.21.15.50:0/2997956452 shutdown_connections 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f4f0a0 0x564178f4d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f46cd0 0x564178f4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 --2- 172.21.15.50:0/2997956452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f5d430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 -- 172.21.15.50:0/2997956452 >> 172.21.15.50:0/2997956452 conn(0x564178f42920 msgr2=0x564178f42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 -- 172.21.15.50:0/2997956452 shutdown_connections 2024-09-12T03:29:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.429+0000 7feb16883740 1 -- 172.21.15.50:0/2997956452 wait complete. 2024-09-12T03:29:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.430+0000 7feb16883740 1 -- start start 2024-09-12T03:29:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.430+0000 7feb16883740 1 Processor -- start 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f59ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f4f0a0 0x564178f4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b3ab0 con 0x564178f4f0a0 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b4f20 con 0x564178f463b0 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c1b4180 con 0x564178f46cd0 2024-09-12T03:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb16883740 10 client.0 _ll_register_callbacks cb 0x7feb040099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb0be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f59ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb11000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb0be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f59ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56400/0 (socket says 172.21.15.50:56400) 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb0be00640 1 -- 172.21.15.50:0/1134902428 learned_addr learned my addr 172.21.15.50:0/1134902428 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb11a00640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f4f0a0 0x564178f4ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.431+0000 7feb0be00640 1 -- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 msgr2=0x564178f595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb0be00640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb0be00640 1 -- 172.21.15.50:0/1134902428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f4f0a0 msgr2=0x564178f4ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb0be00640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f4f0a0 0x564178f4ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb0be00640 1 -- 172.21.15.50:0/1134902428 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564178f4b1a0 con 0x564178f46cd0 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb11a00640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f4f0a0 0x564178f4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb0be00640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f59ae0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaf8098760 tx=0x7feaf809c600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e55464a9f9ebd483 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feaf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feaf2a00640 1 -- 172.21.15.50:0/1134902428 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80a0140 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feaf2a00640 1 -- 172.21.15.50:0/1134902428 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7feaf80cd240 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb000046c0 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feaf2a00640 1 -- 172.21.15.50:0/1134902428 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feaf80cd440 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb16883740 10 client.25402 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.432+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564178fcb760 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feaf2a00640 1 -- 172.21.15.50:0/1134902428 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb000046c0 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feaf2a00640 1 -- 172.21.15.50:0/1134902428 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feaf8018b00 con 0x564178f46cd0 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feb16883740 10 client.25402 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feb16883740 10 client.25402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feb16883740 20 client.25402 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feb16883740 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178fcbed0 0x564178fce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feb16883740 4 client.25402 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.433+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564178eb3b80 con 0x564178fcbed0 2024-09-12T03:29:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.434+0000 7feb11000640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178fcbed0 0x564178fce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.434+0000 7feb11000640 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178fcbed0 0x564178fce2f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe9f85c73b982a0 server_cookie=71944f27b961d4aa in_seq=0 out_seq=0 2024-09-12T03:29:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.434+0000 7feaf2a00640 10 client.25402 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.706+0000 7feaf2a00640 1 -- 172.21.15.50:0/1134902428 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564178eb3b80 con 0x564178fcbed0 2024-09-12T03:29:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.706+0000 7feaf2a00640 10 client.25402 handle_command_reply: tid=0 2024-09-12T03:29:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.706+0000 7feb16883740 1 client.25402 shutdown 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.706+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 msgr2=0x564178f59ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.706+0000 7feb16883740 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f59ae0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaf8098760 tx=0x7feaf809c600 comp rx=0 tx=0).stop 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 shutdown_connections 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564178f4f0a0 0x564178f4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564178fcbed0 0x564178fce2f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564178f46cd0 0x564178f59ae0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 --2- 172.21.15.50:0/1134902428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564178f463b0 0x564178f595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feaf2a00640 0 client.25402 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 >> 172.21.15.50:0/1134902428 conn(0x564178f42920 msgr2=0x564178f4ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 shutdown_connections 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 1 -- 172.21.15.50:0/1134902428 wait complete. 2024-09-12T03:29:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.707+0000 7feb16883740 20 client.25402 trim_cache size 0 max 16384 2024-09-12T03:29:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:29:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:29:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:29:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7feaf40a2cd0 msgr2=0x7feaf40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7feaf40a2cd0 0x7feaf40a5190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb04005fd0 tx=0x7feb04003e50 comp rx=0 tx=0).stop 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1011f0 msgr2=0x7feb0c1a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1011f0 0x7feb0c1a4af0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9f0 tx=0x7feaf800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 shutdown_connections 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb0c105cd0 0x7feb0c19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c1011f0 0x7feb0c1a4af0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7feaf40a2cd0 0x7feaf40a5190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 --2- 172.21.15.50:0/3624168250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1008d0 0x7feb0c1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 >> 172.21.15.50:0/3624168250 conn(0x7feb0c0777c0 msgr2=0x7feb0c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 shutdown_connections 2024-09-12T03:29:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:17.708+0000 7feb12400640 1 -- 172.21.15.50:0/3624168250 wait complete. 2024-09-12T03:29:17.887 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] growfiles 2024-09-12T03:29:17.980 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] iogen 2024-09-12T03:29:18.502 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-stress 2024-09-12T03:29:18.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:18 smithi082 ceph-mon[35046]: pgmap v67: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 542 op/s 2024-09-12T03:29:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:18 smithi050 ceph-mon[30378]: pgmap v67: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 542 op/s 2024-09-12T03:29:18.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:18 smithi142 ceph-mon[35329]: pgmap v67: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 542 op/s 2024-09-12T03:29:19.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:19 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:19.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:19 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:19.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:19 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:20.217 INFO:tasks.workunit.client.0.smithi050.stdout: [DEP] 2024-09-12T03:29:20.218 INFO:tasks.workunit.client.0.smithi050.stdout: [DEP] 2024-09-12T03:29:20.218 INFO:tasks.workunit.client.0.smithi050.stdout: [DEP] 2024-09-12T03:29:20.219 INFO:tasks.workunit.client.0.smithi050.stdout: [DEP] 2024-09-12T03:29:20.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:20 smithi082 ceph-mon[35046]: pgmap v68: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-09-12T03:29:20.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:20 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:20.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:20 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:20 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:29:20.634 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dirstress 2024-09-12T03:29:20.634 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fill 2024-09-12T03:29:20.635 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fill2 2024-09-12T03:29:20.636 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] getpagesize 2024-09-12T03:29:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:20 smithi050 ceph-mon[30378]: pgmap v68: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-09-12T03:29:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:20 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:20 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:20 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:29:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:20 smithi142 ceph-mon[35329]: pgmap v68: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-09-12T03:29:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:20 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:20 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:20 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:29:20.930 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] holes 2024-09-12T03:29:20.943 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] lstat64 2024-09-12T03:29:21.050 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] nametest 2024-09-12T03:29:21.052 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] permname 2024-09-12T03:29:21.272 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] randholes 2024-09-12T03:29:21.314 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] runas 2024-09-12T03:29:21.425 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] truncfile 2024-09-12T03:29:21.461 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] usemem 2024-09-12T03:29:21.663 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] mmapcat 2024-09-12T03:29:21.725 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] append_reader 2024-09-12T03:29:21.749 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] append_writer 2024-09-12T03:29:21.769 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dirperf 2024-09-12T03:29:21.954 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] metaperf 2024-09-12T03:29:22.025 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] devzero 2024-09-12T03:29:22.068 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] feature 2024-09-12T03:29:22.177 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] alloc 2024-09-12T03:29:22.385 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fault 2024-09-12T03:29:22.419 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fstest 2024-09-12T03:29:22.428 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_access_root 2024-09-12T03:29:22.541 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] godown 2024-09-12T03:29:22.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:22 smithi082 ceph-mon[35046]: pgmap v69: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-09-12T03:29:22.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:22 smithi050 ceph-mon[30378]: pgmap v69: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-09-12T03:29:22.697 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] resvtest 2024-09-12T03:29:22.750 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] writemod 2024-09-12T03:29:22.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:22 smithi142 ceph-mon[35329]: pgmap v69: 129 pgs: 129 active+clean; 50 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-09-12T03:29:22.861 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] writev_on_pagefault 2024-09-12T03:29:22.866 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] makeextents 2024-09-12T03:29:23.023 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] itrash 2024-09-12T03:29:23.054 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] rename 2024-09-12T03:29:23.169 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] multi_open_unlink 2024-09-12T03:29:23.227 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] unwritten_sync 2024-09-12T03:29:23.330 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] genhashnames 2024-09-12T03:29:23.332 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_holes 2024-09-12T03:29:23.490 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_writev 2024-09-12T03:29:23.558 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_truncate_cmtime 2024-09-12T03:29:23.632 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dirhash_collide 2024-09-12T03:29:23.691 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_rename_overwrite 2024-09-12T03:29:23.771 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] holetest 2024-09-12T03:29:23.870 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_truncate_self 2024-09-12T03:29:23.971 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] af_unix 2024-09-12T03:29:23.987 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_stale_pmd 2024-09-12T03:29:24.150 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_cow_race 2024-09-12T03:29:24.194 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_fallocate 2024-09-12T03:29:24.278 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fsync-err 2024-09-12T03:29:24.285 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_write_ro 2024-09-12T03:29:24.474 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-12T03:29:24.489 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-12T03:29:24.589 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_ofd_locks 2024-09-12T03:29:24.942 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:24 smithi142 ceph-mon[35329]: pgmap v70: 129 pgs: 129 active+clean; 42 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.2 MiB/s wr, 290 op/s 2024-09-12T03:29:24.942 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:24 smithi082 ceph-mon[35046]: pgmap v70: 129 pgs: 129 active+clean; 42 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.2 MiB/s wr, 290 op/s 2024-09-12T03:29:24.943 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_collision 2024-09-12T03:29:24.943 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:24 smithi050 ceph-mon[30378]: pgmap v70: 129 pgs: 129 active+clean; 42 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.2 MiB/s wr, 290 op/s 2024-09-12T03:29:24.943 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] mmap-write-concurrent 2024-09-12T03:29:24.944 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_get_file_time 2024-09-12T03:29:24.944 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_create_short_dirs 2024-09-12T03:29:25.014 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_create_long_dirs 2024-09-12T03:29:25.120 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_enospc 2024-09-12T03:29:25.121 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_writev_overlap 2024-09-12T03:29:25.288 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] checkpoint_journal 2024-09-12T03:29:25.326 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] mmap-rw-fault 2024-09-12T03:29:25.442 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] allocstale 2024-09-12T03:29:25.456 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_cow_memory_failure 2024-09-12T03:29:25.598 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fake-dump-rootino 2024-09-12T03:29:25.686 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dio-buf-fault 2024-09-12T03:29:25.771 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] rewinddir-test 2024-09-12T03:29:25.791 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] readdir-while-renames 2024-09-12T03:29:25.906 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] xfsctl 2024-09-12T03:29:25.998 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] bstat 2024-09-12T03:29:26.088 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mtab 2024-09-12T03:29:26.107 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] getdevicesize 2024-09-12T03:29:26.288 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] preallo_rw_pattern_reader 2024-09-12T03:29:26.381 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] ftrunc 2024-09-12T03:29:26.381 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] preallo_rw_pattern_writer 2024-09-12T03:29:26.446 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] trunc 2024-09-12T03:29:26.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:26 smithi082 ceph-mon[35046]: pgmap v71: 129 pgs: 129 active+clean; 42 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 220 KiB/s wr, 149 op/s 2024-09-12T03:29:26.609 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fs_perms 2024-09-12T03:29:26.652 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] testx 2024-09-12T03:29:26.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:26 smithi050 ceph-mon[30378]: pgmap v71: 129 pgs: 129 active+clean; 42 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 220 KiB/s wr, 149 op/s 2024-09-12T03:29:26.679 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] looptest 2024-09-12T03:29:26.777 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] locktest 2024-09-12T03:29:26.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:26 smithi142 ceph-mon[35329]: pgmap v71: 129 pgs: 129 active+clean; 42 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 220 KiB/s wr, 149 op/s 2024-09-12T03:29:26.929 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] unwritten_mmap 2024-09-12T03:29:26.930 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] bulkstat_unlink_test 2024-09-12T03:29:27.019 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] deduperace 2024-09-12T03:29:27.246 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] bulkstat_unlink_test_modified 2024-09-12T03:29:27.266 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_dir_offset 2024-09-12T03:29:27.286 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_futimens 2024-09-12T03:29:27.381 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_immutable 2024-09-12T03:29:27.554 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] stale_handle 2024-09-12T03:29:27.567 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] pwrite_mmap_blocked 2024-09-12T03:29:27.610 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_dir_offset2 2024-09-12T03:29:27.872 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] seek_copy_test 2024-09-12T03:29:27.873 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] seek_sanity_test 2024-09-12T03:29:27.954 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_readdir_1 2024-09-12T03:29:28.192 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_readdir_2 2024-09-12T03:29:28.272 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fsync-tester 2024-09-12T03:29:28.294 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] nsexec 2024-09-12T03:29:28.484 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] cloner 2024-09-12T03:29:28.539 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] renameat2 2024-09-12T03:29:28.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:28 smithi082 ceph-mon[35046]: pgmap v72: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 297 KiB/s wr, 164 op/s 2024-09-12T03:29:28.604 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_getcwd 2024-09-12T03:29:28.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:28 smithi050 ceph-mon[30378]: pgmap v72: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 297 KiB/s wr, 164 op/s 2024-09-12T03:29:28.728 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] e4compact 2024-09-12T03:29:28.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:28 smithi142 ceph-mon[35329]: pgmap v72: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 297 KiB/s wr, 164 op/s 2024-09-12T03:29:28.822 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] test-nextquota 2024-09-12T03:29:28.878 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] punch-alternating 2024-09-12T03:29:28.910 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-12T03:29:29.098 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] listxattr 2024-09-12T03:29:29.131 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dio-interleaved 2024-09-12T03:29:29.214 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_dir_type 2024-09-12T03:29:29.243 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] dio-invalidate-cache 2024-09-12T03:29:29.383 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] stat_test 2024-09-12T03:29:29.448 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_encrypted_d_revalidate 2024-09-12T03:29:29.545 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] attr_replace_test 2024-09-12T03:29:29.618 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] swapon 2024-09-12T03:29:29.767 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] mkswap 2024-09-12T03:29:29.871 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_open_tmpfiles 2024-09-12T03:29:29.871 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_attr_corruption 2024-09-12T03:29:29.926 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fscrypt-crypt-util 2024-09-12T03:29:30.083 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] bulkstat_null_ocount 2024-09-12T03:29:30.161 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] splice-test 2024-09-12T03:29:30.204 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] chprojid_fail 2024-09-12T03:29:30.391 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] detached_mounts_propagation 2024-09-12T03:29:30.500 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] ext4_resize 2024-09-12T03:29:30.534 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_readdir_3 2024-09-12T03:29:30.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:30 smithi082 ceph-mon[35046]: pgmap v73: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 198 KiB/s wr, 135 op/s 2024-09-12T03:29:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:30 smithi050 ceph-mon[30378]: pgmap v73: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 198 KiB/s wr, 135 op/s 2024-09-12T03:29:30.764 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] splice2pipe 2024-09-12T03:29:30.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:30 smithi142 ceph-mon[35329]: pgmap v73: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 198 KiB/s wr, 135 op/s 2024-09-12T03:29:30.842 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] uuid_ioctl 2024-09-12T03:29:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 -- 172.21.15.50:0/2298476331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c1048e0 msgr2=0x7f448c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 --2- 172.21.15.50:0/2298476331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c1048e0 0x7f448c108920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f447c009900 tx=0x7f447c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 -- 172.21.15.50:0/2298476331 shutdown_connections 2024-09-12T03:29:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 --2- 172.21.15.50:0/2298476331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c109080 0x7f448c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 --2- 172.21.15.50:0/2298476331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c1048e0 0x7f448c108920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 --2- 172.21.15.50:0/2298476331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c103f30 0x7f448c104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 -- 172.21.15.50:0/2298476331 >> 172.21.15.50:0/2298476331 conn(0x7f448c0ff980 msgr2=0x7f448c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 -- 172.21.15.50:0/2298476331 shutdown_connections 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.864+0000 7f4494800640 1 -- 172.21.15.50:0/2298476331 wait complete. 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 Processor -- start 2024-09-12T03:29:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 -- start start 2024-09-12T03:29:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c103f30 0x7f448c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c109080 0x7f448c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c19ee20 0x7f448c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1140d0 con 0x7f448c103f30 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f448c113f50 con 0x7f448c19ee20 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.866+0000 7f4494800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f448c114250 con 0x7f448c109080 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4491e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c103f30 0x7f448c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4491e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c103f30 0x7f448c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60712/0 (socket says 172.21.15.50:60712) 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4491e00640 1 -- 172.21.15.50:0/2382162618 learned_addr learned my addr 172.21.15.50:0/2382162618 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4492800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c19ee20 0x7f448c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4492800640 1 -- 172.21.15.50:0/2382162618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c109080 msgr2=0x7f448c19aaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4492800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c109080 0x7f448c19aaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4492800640 1 -- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c103f30 msgr2=0x7f448c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4492800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c103f30 0x7f448c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.867+0000 7f4492800640 1 -- 172.21.15.50:0/2382162618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c19f910 con 0x7f448c19ee20 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.868+0000 7f4492800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c19ee20 0x7f448c19f2d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4488009fd0 tx=0x7f448800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.868+0000 7f4486a00640 1 -- 172.21.15.50:0/2382162618 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448800ce20 con 0x7f448c19ee20 2024-09-12T03:29:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.868+0000 7f4486a00640 1 -- 172.21.15.50:0/2382162618 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4488005020 con 0x7f448c19ee20 2024-09-12T03:29:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.868+0000 7f4486a00640 1 -- 172.21.15.50:0/2382162618 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488010720 con 0x7f448c19ee20 2024-09-12T03:29:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.868+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c19fc00 con 0x7f448c19ee20 2024-09-12T03:29:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.868+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c1a73f0 con 0x7f448c19ee20 2024-09-12T03:29:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.873+0000 7f449785f740 1 Processor -- start 2024-09-12T03:29:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.873+0000 7f4486a00640 1 -- 172.21.15.50:0/2382162618 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f44880108c0 con 0x7f448c19ee20 2024-09-12T03:29:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.873+0000 7f449785f740 1 -- start start 2024-09-12T03:29:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.874+0000 7f449785f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.874+0000 7f449785f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.874+0000 7f449785f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cd33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.874+0000 7f449785f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560b7cc8c100 con 0x560b7cd261e0 2024-09-12T03:29:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.874+0000 7f449785f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b7cc341b0 con 0x560b7cd313a0 2024-09-12T03:29:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.874+0000 7f449785f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560b7cc880c0 con 0x560b7cd26b00 2024-09-12T03:29:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.875+0000 7f4491400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.875+0000 7f4491400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd265c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60720/0 (socket says 172.21.15.50:60720) 2024-09-12T03:29:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.875+0000 7f4491400640 1 -- 172.21.15.50:0/2128067944 learned_addr learned my addr 172.21.15.50:0/2128067944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.875+0000 7f4486a00640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f44740a2e20 0x7f44740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.875+0000 7f4491400640 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cd33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.875+0000 7f4486a00640 1 -- 172.21.15.50:0/2382162618 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f44880cfcc0 con 0x7f448c19ee20 2024-09-12T03:29:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4491400640 1 -- 172.21.15.50:0/2128067944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 msgr2=0x560b7cd30e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4491400640 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd30e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4491400640 1 -- 172.21.15.50:0/2128067944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 msgr2=0x560b7cd33790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4491400640 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cd33790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4491400640 1 -- 172.21.15.50:0/2128067944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7c99f210 con 0x560b7cd261e0 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4492800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f44740a2e20 0x7f44740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.876+0000 7f4491400640 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd265c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f447c009230 tx=0x7f447c031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e90851d29e5a891 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.877+0000 7f4492800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f44740a2e20 0x7f44740a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44880a4530 tx=0x7f44880096c0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.877+0000 7f447b000640 1 -- 172.21.15.50:0/2128067944 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c035040 con 0x560b7cd261e0 2024-09-12T03:29:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.877+0000 7f447b000640 1 -- 172.21.15.50:0/2128067944 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c0042d0 con 0x560b7cd261e0 2024-09-12T03:29:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.877+0000 7f447b000640 1 -- 172.21.15.50:0/2128067944 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c0319d0 con 0x560b7cd261e0 2024-09-12T03:29:30.878 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.877+0000 7f449785f740 1 -- 172.21.15.50:0/2128067944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 msgr2=0x560b7cd265c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.877+0000 7f449785f740 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd265c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f447c009230 tx=0x7f447c031510 comp rx=0 tx=0).stop 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 -- 172.21.15.50:0/2128067944 shutdown_connections 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cd33790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 --2- 172.21.15.50:0/2128067944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd265c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 -- 172.21.15.50:0/2128067944 >> 172.21.15.50:0/2128067944 conn(0x560b7cd22750 msgr2=0x560b7cd22b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 -- 172.21.15.50:0/2128067944 shutdown_connections 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 -- 172.21.15.50:0/2128067944 wait complete. 2024-09-12T03:29:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.878+0000 7f449785f740 1 -- start start 2024-09-12T03:29:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 Processor -- start 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd261e0 0x560b7cd2e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd2e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd313a0 0x560b7cd2ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560b7cd33f70 con 0x560b7cd313a0 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b7cc15090 con 0x560b7cd26b00 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560b7cd340f0 con 0x560b7cd261e0 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.879+0000 7f449785f740 10 client.0 _ll_register_callbacks cb 0x560b7cc69f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd2e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd2e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59328/0 (socket says 172.21.15.50:59328) 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 -- 172.21.15.50:0/759503570 learned_addr learned my addr 172.21.15.50:0/759503570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491400640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd313a0 0x560b7cd2ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491400640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd261e0 0x560b7cd2e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 -- 172.21.15.50:0/759503570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd261e0 msgr2=0x560b7cd2e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd261e0 0x560b7cd2e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 -- 172.21.15.50:0/759503570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd313a0 msgr2=0x560b7cd2ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd313a0 0x560b7cd2ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491e00640 1 -- 172.21.15.50:0/759503570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7ca6b620 con 0x560b7cd26b00 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.880+0000 7f4491400640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd313a0 0x560b7cd2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f4491e00640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd2e8c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44800116f0 tx=0x7f4480011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b74eddbe5973a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f446f400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f446f400640 1 -- 172.21.15.50:0/759503570 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448000fb30 con 0x560b7cd26b00 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f446f400640 1 -- 172.21.15.50:0/759503570 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f448000fcd0 con 0x560b7cd26b00 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f447c0042d0 con 0x560b7cd26b00 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f446f400640 1 -- 172.21.15.50:0/759503570 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4480004030 con 0x560b7cd26b00 2024-09-12T03:29:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f449785f740 10 client.25421 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.881+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b7cd3fc60 con 0x560b7cd26b00 2024-09-12T03:29:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.882+0000 7f446f400640 1 -- 172.21.15.50:0/759503570 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c0042d0 con 0x560b7cd26b00 2024-09-12T03:29:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f446f400640 1 -- 172.21.15.50:0/759503570 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f44800035f0 con 0x560b7cd26b00 2024-09-12T03:29:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f449785f740 10 client.25421 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f449785f740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f449785f740 20 client.25421 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f449785f740 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cd411e0 0x560b7cd435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f449785f740 4 client.25421 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b7cc99830 con 0x560b7cd411e0 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.883+0000 7f4491400640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cd411e0 0x560b7cd435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.884+0000 7f4491400640 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cd411e0 0x560b7cd435d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5570d6537083b2a server_cookie=98bf895e2aab790b in_seq=0 out_seq=0 2024-09-12T03:29:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.884+0000 7f446f400640 10 client.25421 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f446f400640 1 -- 172.21.15.50:0/759503570 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b7cc99830 con 0x560b7cd411e0 2024-09-12T03:29:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f446f400640 10 client.25421 handle_command_reply: tid=0 2024-09-12T03:29:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 client.25421 shutdown 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 msgr2=0x560b7cd2e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd2e8c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44800116f0 tx=0x7f4480011bb0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 shutdown_connections 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd313a0 0x560b7cd2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd2e8c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cd411e0 0x560b7cd435d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 --2- 172.21.15.50:0/759503570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd261e0 0x560b7cd2e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 >> 172.21.15.50:0/759503570 conn(0x560b7cd22750 msgr2=0x560b7cd284b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.885+0000 7f446f400640 0 client.25421 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.886+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 shutdown_connections 2024-09-12T03:29:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.886+0000 7f449785f740 1 -- 172.21.15.50:0/759503570 wait complete. 2024-09-12T03:29:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.886+0000 7f449785f740 20 client.25421 trim_cache size 0 max 16384 2024-09-12T03:29:30.894 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fiemap-fault 2024-09-12T03:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.913+0000 7f449785f740 1 Processor -- start 2024-09-12T03:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.913+0000 7f449785f740 1 -- start start 2024-09-12T03:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f449785f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd2ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f449785f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f449785f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 0x560b7cd48e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f449785f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1b39f0 con 0x560b7cd261e0 2024-09-12T03:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f449785f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1b1260 con 0x560b7cd26b00 2024-09-12T03:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f449785f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1af0b0 con 0x560b7cd313a0 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f4491400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 0x560b7cd48e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f4491400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 0x560b7cd48e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:58844/0 (socket says 172.21.15.50:58844) 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f4491400640 1 -- 172.21.15.50:0/2591835231 learned_addr learned my addr 172.21.15.50:0/2591835231 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.914+0000 7f4491400640 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd2ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491400640 1 -- 172.21.15.50:0/2591835231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 msgr2=0x560b7cd301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491e00640 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491400640 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491400640 1 -- 172.21.15.50:0/2591835231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 msgr2=0x560b7cd2ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491400640 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd2ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491400640 1 -- 172.21.15.50:0/2591835231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7cd26010 con 0x560b7cd313a0 2024-09-12T03:29:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.915+0000 7f4491e00640 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.916+0000 7f4491400640 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 0x560b7cd48e90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f447c001eb0 tx=0x7f447c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65425f6a3c0482c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.916+0000 7f447a600640 1 -- 172.21.15.50:0/2591835231 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c035070 con 0x560b7cd313a0 2024-09-12T03:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.916+0000 7f447a600640 1 -- 172.21.15.50:0/2591835231 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c04f040 con 0x560b7cd313a0 2024-09-12T03:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.916+0000 7f447a600640 1 -- 172.21.15.50:0/2591835231 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c003bc0 con 0x560b7cd313a0 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 -- 172.21.15.50:0/2591835231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 msgr2=0x560b7cd48e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 0x560b7cd48e90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f447c001eb0 tx=0x7f447c009260 comp rx=0 tx=0).stop 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 -- 172.21.15.50:0/2591835231 shutdown_connections 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd313a0 0x560b7cd48e90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd26b00 0x560b7cd301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 --2- 172.21.15.50:0/2591835231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd2ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 -- 172.21.15.50:0/2591835231 >> 172.21.15.50:0/2591835231 conn(0x560b7cd22750 msgr2=0x560b7cd2fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.917+0000 7f449785f740 1 -- 172.21.15.50:0/2591835231 shutdown_connections 2024-09-12T03:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.918+0000 7f449785f740 1 -- 172.21.15.50:0/2591835231 wait complete. 2024-09-12T03:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.918+0000 7f449785f740 1 -- start start 2024-09-12T03:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 Processor -- start 2024-09-12T03:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd34400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd34940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cda9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1af720 con 0x560b7cd261e0 2024-09-12T03:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1b0b90 con 0x560b7cd313a0 2024-09-12T03:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1afdf0 con 0x560b7cd26b00 2024-09-12T03:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f449785f740 10 client.0 _ll_register_callbacks cb 0x7f4480009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f4491400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd34400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f4491e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd34940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f4491400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd34400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60736/0 (socket says 172.21.15.50:60736) 2024-09-12T03:29:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.919+0000 7f4491400640 1 -- 172.21.15.50:0/563846401 learned_addr learned my addr 172.21.15.50:0/563846401 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491e00640 1 -- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 msgr2=0x560b7cda9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491400640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cda9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491e00640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cda9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491e00640 1 -- 172.21.15.50:0/563846401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 msgr2=0x560b7cd34400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491e00640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd34400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491e00640 1 -- 172.21.15.50:0/563846401 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7cd35210 con 0x560b7cd26b00 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491400640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd34400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f4491e00640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd34940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x560b7cd33ee0 tx=0x7f4480005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7318d44ff9d40d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.920+0000 7f446f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.921+0000 7f446f400640 1 -- 172.21.15.50:0/563846401 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448002b070 con 0x560b7cd26b00 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.921+0000 7f446f400640 1 -- 172.21.15.50:0/563846401 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f4480005270 con 0x560b7cd26b00 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.921+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f447c04f040 con 0x560b7cd26b00 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.921+0000 7f446f400640 1 -- 172.21.15.50:0/563846401 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4480033050 con 0x560b7cd26b00 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 10 client.25414 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b7cdab470 con 0x560b7cd26b00 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f446f400640 1 -- 172.21.15.50:0/563846401 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c04f040 con 0x560b7cd26b00 2024-09-12T03:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f446f400640 1 -- 172.21.15.50:0/563846401 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f448002e030 con 0x560b7cd26b00 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 10 client.25414 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 10 client.25414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 20 client.25414 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cdabc20 0x560b7cdae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 4 client.25414 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.922+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b7cc93b80 con 0x560b7cdabc20 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.923+0000 7f4491400640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cdabc20 0x560b7cdae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.923+0000 7f446f400640 10 client.25414 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.923+0000 7f4491400640 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cdabc20 0x560b7cdae040 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd0b49a0ac0bb67 server_cookie=a19be70afe4e1d24 in_seq=0 out_seq=0 2024-09-12T03:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.944+0000 7f446f400640 1 -- 172.21.15.50:0/563846401 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b7cc93b80 con 0x560b7cdabc20 2024-09-12T03:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.944+0000 7f446f400640 10 client.25414 handle_command_reply: tid=0 2024-09-12T03:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.944+0000 7f449785f740 1 client.25414 shutdown 2024-09-12T03:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.944+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 msgr2=0x560b7cd34940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.944+0000 7f449785f740 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd34940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x560b7cd33ee0 tx=0x7f4480005cc0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 shutdown_connections 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b7cd313a0 0x560b7cda9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x560b7cdabc20 0x560b7cdae040 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b7cd26b00 0x560b7cd34940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 --2- 172.21.15.50:0/563846401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560b7cd261e0 0x560b7cd34400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f446f400640 0 client.25414 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 >> 172.21.15.50:0/563846401 conn(0x560b7cd22750 msgr2=0x560b7cd217a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 shutdown_connections 2024-09-12T03:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 1 -- 172.21.15.50:0/563846401 wait complete. 2024-09-12T03:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.945+0000 7f449785f740 20 client.25414 trim_cache size 0 max 16384 2024-09-12T03:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.946+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f44740a2e20 msgr2=0x7f44740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.946+0000 7f4494800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f44740a2e20 0x7f44740a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44880a4530 tx=0x7f44880096c0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.946+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c19ee20 msgr2=0x7f448c19f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.946+0000 7f4494800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c19ee20 0x7f448c19f2d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4488009fd0 tx=0x7f448800ef10 comp rx=0 tx=0).stop 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.947+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 shutdown_connections 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.947+0000 7f4494800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f448c19ee20 0x7f448c19f2d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.947+0000 7f4494800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448c109080 0x7f448c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.947+0000 7f4494800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f44740a2e20 0x7f44740a52e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.947+0000 7f4494800640 1 --2- 172.21.15.50:0/2382162618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f448c103f30 0x7f448c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.947+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 >> 172.21.15.50:0/2382162618 conn(0x7f448c0ff980 msgr2=0x7f448c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.948+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 shutdown_connections 2024-09-12T03:29:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:30.948+0000 7f4494800640 1 -- 172.21.15.50:0/2382162618 wait complete. 2024-09-12T03:29:31.098 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fiemap-tester 2024-09-12T03:29:31.183 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_stripealign 2024-09-12T03:29:31.189 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] open_by_handle 2024-09-12T03:29:31.197 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_mmap_dio 2024-09-12T03:29:31.485 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] t_reflink_read_race 2024-09-12T03:29:31.485 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] xfsfind 2024-09-12T03:29:31.525 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] fssum 2024-09-12T03:29:31.601 INFO:tasks.workunit.client.0.smithi050.stdout:Building log-writes 2024-09-12T03:29:31.616 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] replay-log 2024-09-12T03:29:31.836 INFO:tasks.workunit.client.0.smithi050.stdout:Building perf 2024-09-12T03:29:31.844 INFO:tasks.workunit.client.0.smithi050.stdout:Building aio-dio-regress 2024-09-12T03:29:31.845 INFO:tasks.workunit.client.0.smithi050.stdout:Building vfs 2024-09-12T03:29:31.857 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] vfstest 2024-09-12T03:29:31.857 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aiocp 2024-09-12T03:29:32.130 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-12T03:29:32.147 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] mount-idmapped 2024-09-12T03:29:32.239 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-append-write-read-race 2024-09-12T03:29:32.425 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-cow-race 2024-09-12T03:29:32.565 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-cycle-write 2024-09-12T03:29:32.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:32 smithi082 ceph-mon[35046]: pgmap v74: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 198 KiB/s wr, 135 op/s 2024-09-12T03:29:32.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:32 smithi050 ceph-mon[30378]: pgmap v74: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 198 KiB/s wr, 135 op/s 2024-09-12T03:29:32.714 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-eof-race 2024-09-12T03:29:32.731 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-extend-stat 2024-09-12T03:29:32.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:32 smithi142 ceph-mon[35329]: pgmap v74: 129 pgs: 129 active+clean; 43 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 198 KiB/s wr, 135 op/s 2024-09-12T03:29:32.928 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-fcntl-race 2024-09-12T03:29:33.037 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-hole-filling-race 2024-09-12T03:29:33.040 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-invalidate-failure 2024-09-12T03:29:33.256 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-invalidate-readahead 2024-09-12T03:29:33.330 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aiodio_sparse2 2024-09-12T03:29:33.335 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-subblock-eof-read 2024-09-12T03:29:33.571 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-dio-write-verify 2024-09-12T03:29:33.619 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-12T03:29:33.687 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-12T03:29:33.875 INFO:tasks.workunit.client.0.smithi050.stdout: [CC] aio-last-ref-held-by-io 2024-09-12T03:29:34.556 INFO:tasks.workunit.client.0.smithi050.stderr:+ cd .. 2024-09-12T03:29:34.557 INFO:tasks.workunit.client.0.smithi050.stderr:+ cp xfstests-dev/ltp/fsx . 2024-09-12T03:29:34.562 INFO:tasks.workunit.client.0.smithi050.stderr:+ OPTIONS=-z 2024-09-12T03:29:34.562 INFO:tasks.workunit.client.0.smithi050.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-09-12T03:29:34.564 INFO:tasks.workunit.client.0.smithi050.stdout:Seed set to 1 2024-09-12T03:29:34.590 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T03:29:34.590 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T03:29:34.592 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T03:29:34.592 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T03:29:34.593 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T03:29:34.594 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T03:29:34.595 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:34 smithi142 ceph-mon[35329]: pgmap v75: 129 pgs: 129 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 273 KiB/s wr, 154 op/s 2024-09-12T03:29:34.596 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T03:29:34.597 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero size read 2024-09-12T03:29:34.597 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero size read 2024-09-12T03:29:34.597 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x81d54 2024-09-12T03:29:34.601 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xb86d7 2024-09-12T03:29:34.603 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:34 smithi082 ceph-mon[35046]: pgmap v75: 129 pgs: 129 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 273 KiB/s wr, 154 op/s 2024-09-12T03:29:34.611 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xe627e 2024-09-12T03:29:34.662 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0xf3194 2024-09-12T03:29:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:34 smithi050 ceph-mon[30378]: pgmap v75: 129 pgs: 129 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 273 KiB/s wr, 154 op/s 2024-09-12T03:29:34.690 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xf3ceb 2024-09-12T03:29:34.697 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0xf47e4 2024-09-12T03:29:34.822 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xfaccc 2024-09-12T03:29:35.408 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xfe423 2024-09-12T03:29:36.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:36 smithi082 ceph-mon[35046]: pgmap v76: 129 pgs: 129 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 152 KiB/s wr, 33 op/s 2024-09-12T03:29:36.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:36 smithi050 ceph-mon[30378]: pgmap v76: 129 pgs: 129 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 152 KiB/s wr, 33 op/s 2024-09-12T03:29:36.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:36 smithi142 ceph-mon[35329]: pgmap v76: 129 pgs: 129 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 152 KiB/s wr, 33 op/s 2024-09-12T03:29:38.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:38 smithi082 ceph-mon[35046]: pgmap v77: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 949 KiB/s wr, 163 op/s 2024-09-12T03:29:38.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:38 smithi050 ceph-mon[30378]: pgmap v77: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 949 KiB/s wr, 163 op/s 2024-09-12T03:29:38.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:38 smithi142 ceph-mon[35329]: pgmap v77: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 949 KiB/s wr, 163 op/s 2024-09-12T03:29:38.835 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xfe492 2024-09-12T03:29:40.502 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:40 smithi050 ceph-mon[30378]: pgmap v78: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 872 KiB/s wr, 148 op/s 2024-09-12T03:29:40.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:40 smithi082 ceph-mon[35046]: pgmap v78: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 872 KiB/s wr, 148 op/s 2024-09-12T03:29:40.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:40 smithi142 ceph-mon[35329]: pgmap v78: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 872 KiB/s wr, 148 op/s 2024-09-12T03:29:42.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:42 smithi082 ceph-mon[35046]: pgmap v79: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 872 KiB/s wr, 148 op/s 2024-09-12T03:29:42.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:42 smithi050 ceph-mon[30378]: pgmap v79: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 872 KiB/s wr, 148 op/s 2024-09-12T03:29:42.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:42 smithi142 ceph-mon[35329]: pgmap v79: 129 pgs: 129 active+clean; 45 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 872 KiB/s wr, 148 op/s 2024-09-12T03:29:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 -- 172.21.15.50:0/1755846454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 msgr2=0x7fceec1109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 --2- 172.21.15.50:0/1755846454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec1109d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009be0 tx=0x7fcee402a6c0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 -- 172.21.15.50:0/1755846454 shutdown_connections 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 --2- 172.21.15.50:0/1755846454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec1109d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 --2- 172.21.15.50:0/1755846454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec103b00 0x7fceec103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 --2- 172.21.15.50:0/1755846454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceec109ba0 0x7fceec109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.092+0000 7fcef2400640 1 -- 172.21.15.50:0/1755846454 >> 172.21.15.50:0/1755846454 conn(0x7fceec0ff7c0 msgr2=0x7fceec101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.093+0000 7fcef2400640 1 -- 172.21.15.50:0/1755846454 shutdown_connections 2024-09-12T03:29:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.093+0000 7fcef2400640 1 -- 172.21.15.50:0/1755846454 wait complete. 2024-09-12T03:29:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 Processor -- start 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 -- start start 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceec103b00 0x7fceec19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 0x7fceec19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceec113f10 con 0x7fceec103b00 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceec113d90 con 0x7fceec1044a0 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef2400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fceec114090 con 0x7fceec109ba0 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.094+0000 7fcef1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceec103b00 0x7fceec19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fcef1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 0x7fceec19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fcef1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 0x7fceec19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52816/0 (socket says 172.21.15.50:52816) 2024-09-12T03:29:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fcef1a00640 1 -- 172.21.15.50:0/93680480 learned_addr learned my addr 172.21.15.50:0/93680480 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 -- 172.21.15.50:0/93680480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 msgr2=0x7fceec19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 0x7fceec19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 -- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceec103b00 msgr2=0x7fceec19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceec103b00 0x7fceec19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 -- 172.21.15.50:0/93680480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec19f3d0 con 0x7fceec1044a0 2024-09-12T03:29:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fcef1a00640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 0x7fceec19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.095+0000 7fceebe00640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec19a920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fced8005fd0 tx=0x7fced800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.096+0000 7fcee9600640 1 -- 172.21.15.50:0/93680480 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced800cd00 con 0x7fceec1044a0 2024-09-12T03:29:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.096+0000 7fcee9600640 1 -- 172.21.15.50:0/93680480 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced80035f0 con 0x7fceec1044a0 2024-09-12T03:29:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.096+0000 7fcee9600640 1 -- 172.21.15.50:0/93680480 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8010690 con 0x7fceec1044a0 2024-09-12T03:29:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.096+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec19f6c0 con 0x7fceec1044a0 2024-09-12T03:29:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.096+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec1a6f50 con 0x7fceec1044a0 2024-09-12T03:29:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.099+0000 7fcef67e0740 1 Processor -- start 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcee9600640 1 -- 172.21.15.50:0/93680480 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fced80037a0 con 0x7fceec1044a0 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 -- start start 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c86bd0 0x563584c99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c99d80 0x563584c9c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563584bec100 con 0x563584c99d80 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563584b941b0 con 0x563584c862b0 2024-09-12T03:29:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef67e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563584be80c0 con 0x563584c86bd0 2024-09-12T03:29:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c86690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59836/0 (socket says 172.21.15.50:59836) 2024-09-12T03:29:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c86bd0 0x563584c99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.100+0000 7fcef1a00640 1 -- 172.21.15.50:0/3434542077 learned_addr learned my addr 172.21.15.50:0/3434542077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.101+0000 7fcef1a00640 1 -- 172.21.15.50:0/3434542077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c86bd0 msgr2=0x563584c99840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.101+0000 7fcef1a00640 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c86bd0 0x563584c99840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.101+0000 7fcef1a00640 1 -- 172.21.15.50:0/3434542077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c99d80 msgr2=0x563584c9c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.101+0000 7fceebe00640 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c99d80 0x563584c9c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.101+0000 7fcef1a00640 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c99d80 0x563584c9c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcef1a00640 1 -- 172.21.15.50:0/3434542077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635848ff210 con 0x563584c862b0 2024-09-12T03:29:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcee9600640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fced40a2cf0 0x7fced40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fceebe00640 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c99d80 0x563584c9c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcee9600640 1 -- 172.21.15.50:0/93680480 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fced80cfa80 con 0x7fceec1044a0 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcef1000640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fced40a2cf0 0x7fced40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcef1a00640 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c86690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcee4032fd0 tx=0x7fcee40098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27f716c2fe4c7af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcede000640 1 -- 172.21.15.50:0/3434542077 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4041070 con 0x563584c862b0 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.102+0000 7fcede000640 1 -- 172.21.15.50:0/3434542077 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee4003e50 con 0x563584c862b0 2024-09-12T03:29:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcede000640 1 -- 172.21.15.50:0/3434542077 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4036040 con 0x563584c862b0 2024-09-12T03:29:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 -- 172.21.15.50:0/3434542077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 msgr2=0x563584c86690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c86690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcee4032fd0 tx=0x7fcee40098e0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef1000640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fced40a2cf0 0x7fced40a51b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x563584c88f80 tx=0x7fcee0005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 -- 172.21.15.50:0/3434542077 shutdown_connections 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c99d80 0x563584c9c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c86bd0 0x563584c99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 --2- 172.21.15.50:0/3434542077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c86690 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.103+0000 7fcef67e0740 1 -- 172.21.15.50:0/3434542077 >> 172.21.15.50:0/3434542077 conn(0x563584c82920 msgr2=0x563584c82d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.104+0000 7fcef67e0740 1 -- 172.21.15.50:0/3434542077 shutdown_connections 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.104+0000 7fcef67e0740 1 -- 172.21.15.50:0/3434542077 wait complete. 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.104+0000 7fcef67e0740 1 -- start start 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.104+0000 7fcef67e0740 1 Processor -- start 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584c97230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584c97770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563584c9c980 con 0x563584c862b0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563584b75090 con 0x563584c86bd0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563584c9cb00 con 0x563584c99d80 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef67e0740 10 client.0 _ll_register_callbacks cb 0x563584bc9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fceebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584c97230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fceebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584c97230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59842/0 (socket says 172.21.15.50:59842) 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fceebe00640 1 -- 172.21.15.50:0/879981077 learned_addr learned my addr 172.21.15.50:0/879981077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.105+0000 7fcef1a00640 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584c97770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fceebe00640 1 -- 172.21.15.50:0/879981077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 msgr2=0x563584c97770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fceebe00640 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584c97770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fceebe00640 1 -- 172.21.15.50:0/879981077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 msgr2=0x563584c96cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fceebe00640 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c96cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fceebe00640 1 -- 172.21.15.50:0/879981077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635849cb620 con 0x563584c86bd0 2024-09-12T03:29:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fceebe00640 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584c97230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced809b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00fa009036c3319 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fced2a00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fced2a00640 1 -- 172.21.15.50:0/879981077 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80cd260 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fced2a00640 1 -- 172.21.15.50:0/879981077 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fced809b000 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee4003e50 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.106+0000 7fced2a00640 1 -- 172.21.15.50:0/879981077 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fced809ff60 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 10 client.25433 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563584ca8630 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fced2a00640 1 -- 172.21.15.50:0/879981077 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcee4003e50 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fced2a00640 1 -- 172.21.15.50:0/879981077 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fced809aa70 con 0x563584c86bd0 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 10 client.25433 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 20 client.25433 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584ca9af0 0x563584cabee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 4 client.25433 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563584bf9830 con 0x563584ca9af0 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.107+0000 7fcef1000640 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584ca9af0 0x563584cabee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.108+0000 7fcef1000640 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584ca9af0 0x563584cabee0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9fadfad2337f62 server_cookie=823ac41eb75a9d71 in_seq=0 out_seq=0 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.108+0000 7fced2a00640 10 client.25433 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fced2a00640 1 -- 172.21.15.50:0/879981077 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563584bf9830 con 0x563584ca9af0 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fced2a00640 10 client.25433 handle_command_reply: tid=0 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 client.25433 shutdown 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 msgr2=0x563584c97230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584c97230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced809b600 comp rx=0 tx=0).stop 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 shutdown_connections 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584ca9af0 0x563584cabee0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584c97770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584c97230 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 --2- 172.21.15.50:0/879981077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 >> 172.21.15.50:0/879981077 conn(0x563584c82920 msgr2=0x563584c88680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fced2a00640 0 client.25433 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 shutdown_connections 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 1 -- 172.21.15.50:0/879981077 wait complete. 2024-09-12T03:29:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.109+0000 7fcef67e0740 20 client.25433 trim_cache size 0 max 16384 2024-09-12T03:29:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.127+0000 7fcef67e0740 1 Processor -- start 2024-09-12T03:29:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 -- start start 2024-09-12T03:29:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c98b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584cb1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584cb1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1af5b0 con 0x563584c862b0 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1ace20 con 0x563584c86bd0 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef67e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1aac70 con 0x563584c99d80 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584cb1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c98b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584cb1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52838/0 (socket says 172.21.15.50:52838) 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fcef1a00640 1 -- 172.21.15.50:0/4129353556 learned_addr learned my addr 172.21.15.50:0/4129353556 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.128+0000 7fceebe00640 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584cb1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fceebe00640 1 -- 172.21.15.50:0/4129353556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 msgr2=0x563584cb1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fceebe00640 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584cb1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fceebe00640 1 -- 172.21.15.50:0/4129353556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 msgr2=0x563584c98b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fceebe00640 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c98b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fceebe00640 1 -- 172.21.15.50:0/4129353556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563584c821b0 con 0x563584c86bd0 2024-09-12T03:29:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fcef1000640 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c98b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fceebe00640 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584cb1780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fced8098010 tx=0x7fced809bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1affd7d0b968084 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.129+0000 7fcedd600640 1 -- 172.21.15.50:0/4129353556 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced809aa70 con 0x563584c86bd0 2024-09-12T03:29:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcedd600640 1 -- 172.21.15.50:0/4129353556 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced809ac10 con 0x563584c86bd0 2024-09-12T03:29:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcedd600640 1 -- 172.21.15.50:0/4129353556 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80a14b0 con 0x563584c86bd0 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 -- 172.21.15.50:0/4129353556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 msgr2=0x563584cb1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584cb1780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fced8098010 tx=0x7fced809bd30 comp rx=0 tx=0).stop 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 -- 172.21.15.50:0/4129353556 shutdown_connections 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584cb1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c86bd0 0x563584cb1780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4129353556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c862b0 0x563584c98b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.130+0000 7fcef67e0740 1 -- 172.21.15.50:0/4129353556 >> 172.21.15.50:0/4129353556 conn(0x563584c82920 msgr2=0x563584c98440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.131+0000 7fcef67e0740 1 -- 172.21.15.50:0/4129353556 shutdown_connections 2024-09-12T03:29:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.131+0000 7fcef67e0740 1 -- 172.21.15.50:0/4129353556 wait complete. 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.131+0000 7fcef67e0740 1 -- start start 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.131+0000 7fcef67e0740 1 Processor -- start 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c9d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c86bd0 0x563584c9da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584d0c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1ab2e0 con 0x563584c86bd0 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1ac750 con 0x563584c862b0 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1ab9b0 con 0x563584c99d80 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef67e0740 10 client.0 _ll_register_callbacks cb 0x7fced8003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c9d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c9d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59848/0 (socket says 172.21.15.50:59848) 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 -- 172.21.15.50:0/4155446069 learned_addr learned my addr 172.21.15.50:0/4155446069 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fceebe00640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c86bd0 0x563584c9da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1a00640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584d0c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 -- 172.21.15.50:0/4155446069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 msgr2=0x563584d0c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584d0c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 -- 172.21.15.50:0/4155446069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c86bd0 msgr2=0x563584c9da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c86bd0 0x563584c9da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1000640 1 -- 172.21.15.50:0/4155446069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563584d0cdb0 con 0x563584c862b0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.132+0000 7fcef1a00640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584d0c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fcef1000640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c9d500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcee0012cb0 tx=0x7fcee00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd07553924be8255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fced2a00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fced2a00640 1 -- 172.21.15.50:0/4155446069 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0003db0 con 0x563584c862b0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fced2a00640 1 -- 172.21.15.50:0/4155446069 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fcee002b380 con 0x563584c862b0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced809ac10 con 0x563584c862b0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fced2a00640 1 -- 172.21.15.50:0/4155446069 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcee003c050 con 0x563584c862b0 2024-09-12T03:29:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fcef67e0740 10 client.25439 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.133+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563584d14170 con 0x563584c862b0 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fced2a00640 1 -- 172.21.15.50:0/4155446069 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced809ac10 con 0x563584c862b0 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fced2a00640 1 -- 172.21.15.50:0/4155446069 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcee001f8c0 con 0x563584c862b0 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fcef67e0740 10 client.25439 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fcef67e0740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fcef67e0740 20 client.25439 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584d14930 0x563584d16d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fcef67e0740 4 client.25439 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563584bf3b80 con 0x563584d14930 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.134+0000 7fceebe00640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584d14930 0x563584d16d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.135+0000 7fceebe00640 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584d14930 0x563584d16d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7efaf4bf1ffed8b server_cookie=764c0bc91c0504b0 in_seq=0 out_seq=0 2024-09-12T03:29:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.135+0000 7fced2a00640 10 client.25439 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.152+0000 7fced2a00640 1 -- 172.21.15.50:0/4155446069 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563584bf3b80 con 0x563584d14930 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.152+0000 7fced2a00640 10 client.25439 handle_command_reply: tid=0 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.152+0000 7fcef67e0740 1 client.25439 shutdown 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 msgr2=0x563584c9d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c9d500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcee0012cb0 tx=0x7fcee00046c0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 shutdown_connections 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563584d14930 0x563584d16d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563584c99d80 0x563584d0c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563584c86bd0 0x563584c9da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fced2a00640 0 client.25439 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 --2- 172.21.15.50:0/4155446069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563584c862b0 0x563584c9d500 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.153+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 >> 172.21.15.50:0/4155446069 conn(0x563584c82920 msgr2=0x563584c87df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 shutdown_connections 2024-09-12T03:29:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef67e0740 1 -- 172.21.15.50:0/4155446069 wait complete. 2024-09-12T03:29:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef67e0740 20 client.25439 trim_cache size 0 max 16384 2024-09-12T03:29:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fced40a2cf0 msgr2=0x7fced40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef2400640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fced40a2cf0 0x7fced40a51b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x563584c88f80 tx=0x7fcee0005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 msgr2=0x7fceec19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.154+0000 7fcef2400640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec19a920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fced8005fd0 tx=0x7fced800ee50 comp rx=0 tx=0).stop 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 shutdown_connections 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec109ba0 0x7fceec19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec1044a0 0x7fceec19a920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fced40a2cf0 0x7fced40a51b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 --2- 172.21.15.50:0/93680480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fceec103b00 0x7fceec19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 >> 172.21.15.50:0/93680480 conn(0x7fceec0ff7c0 msgr2=0x7fceec0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 shutdown_connections 2024-09-12T03:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:44.155+0000 7fcef2400640 1 -- 172.21.15.50:0/93680480 wait complete. 2024-09-12T03:29:44.250 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xff038 2024-09-12T03:29:44.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:44 smithi082 ceph-mon[35046]: pgmap v80: 129 pgs: 129 active+clean; 48 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-09-12T03:29:44.614 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:44 smithi050 ceph-mon[30378]: pgmap v80: 129 pgs: 129 active+clean; 48 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-09-12T03:29:44.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:44 smithi142 ceph-mon[35329]: pgmap v80: 129 pgs: 129 active+clean; 48 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-09-12T03:29:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:46 smithi082 ceph-mon[35046]: pgmap v81: 129 pgs: 129 active+clean; 48 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-12T03:29:46.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:46 smithi050 ceph-mon[30378]: pgmap v81: 129 pgs: 129 active+clean; 48 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-12T03:29:46.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:46 smithi142 ceph-mon[35329]: pgmap v81: 129 pgs: 129 active+clean; 48 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-12T03:29:46.882 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0xffcac 2024-09-12T03:29:47.048 INFO:tasks.workunit.client.0.smithi050.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-09-12T03:29:48.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:48 smithi050 ceph-mon[30378]: pgmap v82: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 478 op/s 2024-09-12T03:29:48.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:48 smithi142 ceph-mon[35329]: pgmap v82: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 478 op/s 2024-09-12T03:29:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:48 smithi082 ceph-mon[35046]: pgmap v82: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 478 op/s 2024-09-12T03:29:50.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:50 smithi050 ceph-mon[30378]: pgmap v83: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2024-09-12T03:29:50.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:50 smithi142 ceph-mon[35329]: pgmap v83: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2024-09-12T03:29:50.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:50 smithi082 ceph-mon[35046]: pgmap v83: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2024-09-12T03:29:52.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:52 smithi050 ceph-mon[30378]: pgmap v84: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2024-09-12T03:29:52.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:52 smithi142 ceph-mon[35329]: pgmap v84: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2024-09-12T03:29:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:52 smithi082 ceph-mon[35046]: pgmap v84: 129 pgs: 129 active+clean; 46 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2024-09-12T03:29:54.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:54 smithi050 ceph-mon[30378]: pgmap v85: 129 pgs: 129 active+clean; 51 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 558 op/s 2024-09-12T03:29:54.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:54 smithi142 ceph-mon[35329]: pgmap v85: 129 pgs: 129 active+clean; 51 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 558 op/s 2024-09-12T03:29:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:54 smithi082 ceph-mon[35046]: pgmap v85: 129 pgs: 129 active+clean; 51 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 558 op/s 2024-09-12T03:29:56.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:56 smithi050 ceph-mon[30378]: pgmap v86: 129 pgs: 129 active+clean; 51 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 403 op/s 2024-09-12T03:29:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:56 smithi142 ceph-mon[35329]: pgmap v86: 129 pgs: 129 active+clean; 51 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 403 op/s 2024-09-12T03:29:56.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:56 smithi082 ceph-mon[35046]: pgmap v86: 129 pgs: 129 active+clean; 51 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 403 op/s 2024-09-12T03:29:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.290+0000 7f0ccb200640 1 -- 172.21.15.50:0/330327170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 msgr2=0x7f0ccc106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.290+0000 7f0ccb200640 1 --2- 172.21.15.50:0/330327170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 0x7f0ccc106d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009930 tx=0x7f0cbc02a4d0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 -- 172.21.15.50:0/330327170 shutdown_connections 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 --2- 172.21.15.50:0/330327170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 --2- 172.21.15.50:0/330327170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc0ff9d0 0x7f0ccc0ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 --2- 172.21.15.50:0/330327170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 0x7f0ccc106d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 -- 172.21.15.50:0/330327170 >> 172.21.15.50:0/330327170 conn(0x7f0ccc0fb820 msgr2=0x7f0ccc0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 -- 172.21.15.50:0/330327170 shutdown_connections 2024-09-12T03:29:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.291+0000 7f0ccb200640 1 -- 172.21.15.50:0/330327170 wait complete. 2024-09-12T03:29:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 Processor -- start 2024-09-12T03:29:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 -- start start 2024-09-12T03:29:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc0ff9d0 0x7f0ccc195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 0x7f0ccc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10fb20 con 0x7f0ccc0ff9d0 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f9a0 con 0x7f0ccc100390 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0ccb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10fca0 con 0x7f0ccc106960 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0cca800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 0x7f0ccc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:47554/0 (socket says 172.21.15.50:47554) 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0cc3e00640 1 -- 172.21.15.50:0/4117103283 learned_addr learned my addr 172.21.15.50:0/4117103283 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.292+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc0ff9d0 0x7f0ccc195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc3e00640 1 -- 172.21.15.50:0/4117103283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 msgr2=0x7f0ccc19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 0x7f0ccc19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc3e00640 1 -- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc0ff9d0 msgr2=0x7f0ccc195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc0ff9d0 0x7f0ccc195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc3e00640 1 -- 172.21.15.50:0/4117103283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc19af80 con 0x7f0ccc100390 2024-09-12T03:29:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc1964d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9f0 tx=0x7f0cb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc2a00640 1 -- 172.21.15.50:0/4117103283 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400cd80 con 0x7f0ccc100390 2024-09-12T03:29:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc19b270 con 0x7f0ccc100390 2024-09-12T03:29:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc2a00640 1 -- 172.21.15.50:0/4117103283 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb4010910 con 0x7f0ccc100390 2024-09-12T03:29:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.293+0000 7f0cc2a00640 1 -- 172.21.15.50:0/4117103283 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4018990 con 0x7f0ccc100390 2024-09-12T03:29:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.294+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a2b00 con 0x7f0ccc100390 2024-09-12T03:29:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.297+0000 7f0cc2a00640 1 -- 172.21.15.50:0/4117103283 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0cb4010ac0 con 0x7f0ccc100390 2024-09-12T03:29:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.297+0000 7f0cd3a3c740 1 Processor -- start 2024-09-12T03:29:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.297+0000 7f0cd3a3c740 1 -- start start 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1824790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b1831960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5644b178a100 con 0x5644b182f570 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5644b17321b0 con 0x5644b18243b0 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644b17860c0 con 0x5644b1824cd0 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cca800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1824790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cc9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cca800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1824790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:47558/0 (socket says 172.21.15.50:47558) 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cca800640 1 -- 172.21.15.50:0/2817831457 learned_addr learned my addr 172.21.15.50:0/2817831457 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cc9e00640 1 -- 172.21.15.50:0/2817831457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 msgr2=0x5644b1824790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1824790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cc9e00640 1 -- 172.21.15.50:0/2817831457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 msgr2=0x5644b1831960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b1831960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cc9e00640 1 -- 172.21.15.50:0/2817831457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b149d210 con 0x5644b1824cd0 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.298+0000 7f0cca800640 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1824790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cc2a00640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0cb00a7100 0x7f0cb00a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182f030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc02a9f0 tx=0x7f0cbc005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e38f0747f9383b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cc2a00640 1 -- 172.21.15.50:0/4117103283 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f0cb40cfb30 con 0x7f0ccc100390 2024-09-12T03:29:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cca800640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0cb00a7100 0x7f0cb00a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cbb000640 1 -- 172.21.15.50:0/2817831457 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc035040 con 0x5644b1824cd0 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cbb000640 1 -- 172.21.15.50:0/2817831457 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc005520 con 0x5644b1824cd0 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.299+0000 7f0cbb000640 1 -- 172.21.15.50:0/2817831457 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0035f0 con 0x5644b1824cd0 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/2817831457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 msgr2=0x5644b182f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cca800640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0cb00a7100 0x7f0cb00a95c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4011890 tx=0x7f0cc4009500 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182f030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc02a9f0 tx=0x7f0cbc005de0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/2817831457 shutdown_connections 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b1831960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182f030 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/2817831457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1824790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/2817831457 >> 172.21.15.50:0/2817831457 conn(0x5644b1820920 msgr2=0x5644b1820d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/2817831457 shutdown_connections 2024-09-12T03:29:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.300+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/2817831457 wait complete. 2024-09-12T03:29:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.301+0000 7f0cd3a3c740 1 -- start start 2024-09-12T03:29:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.301+0000 7f0cd3a3c740 1 Processor -- start 2024-09-12T03:29:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b18243b0 0x5644b182c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 0x5644b182d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5644b1832140 con 0x5644b18243b0 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5644b1713090 con 0x5644b182f570 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644b18322c0 con 0x5644b1824cd0 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cd3a3c740 10 client.0 _ll_register_callbacks cb 0x5644b1767f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cca800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 0x5644b182d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cca800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 0x5644b182d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:47566/0 (socket says 172.21.15.50:47566) 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cca800640 1 -- 172.21.15.50:0/1339396482 learned_addr learned my addr 172.21.15.50:0/1339396482 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.302+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b18243b0 0x5644b182c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cca800640 1 -- 172.21.15.50:0/1339396482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 msgr2=0x5644b182cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cca800640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cca800640 1 -- 172.21.15.50:0/1339396482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b18243b0 msgr2=0x5644b182c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cca800640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b18243b0 0x5644b182c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cca800640 1 -- 172.21.15.50:0/1339396482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b1569620 con 0x5644b182f570 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b18243b0 0x5644b182c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0cca800640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 0x5644b182d0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400fbe0 tx=0x7f0cc4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7255e61e8197ddd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0caf400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.303+0000 7f0caf400640 1 -- 172.21.15.50:0/1339396482 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400fe50 con 0x5644b182f570 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0caf400640 1 -- 172.21.15.50:0/1339396482 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f0cc4017030 con 0x5644b182f570 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cbc005520 con 0x5644b182f570 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0caf400640 1 -- 172.21.15.50:0/1339396482 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cc4004b20 con 0x5644b182f570 2024-09-12T03:29:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 10 client.25451 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644b184f000 con 0x5644b182f570 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0caf400640 1 -- 172.21.15.50:0/1339396482 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc005520 con 0x5644b182f570 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0caf400640 1 -- 172.21.15.50:0/1339396482 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0cc4004030 con 0x5644b182f570 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 10 client.25451 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 20 client.25451 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b1850510 0x5644b1852960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 4 client.25451 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.304+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b1797830 con 0x5644b1850510 2024-09-12T03:29:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.305+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b1850510 0x5644b1852960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.305+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b1850510 0x5644b1852960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e70c54fa1916a6a server_cookie=be064ecb2827b96f in_seq=0 out_seq=0 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.305+0000 7f0caf400640 10 client.25451 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0caf400640 1 -- 172.21.15.50:0/1339396482 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644b1797830 con 0x5644b1850510 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0caf400640 10 client.25451 handle_command_reply: tid=0 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 client.25451 shutdown 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 msgr2=0x5644b182d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 0x5644b182d0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400fbe0 tx=0x7f0cc4004ab0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 shutdown_connections 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b182f570 0x5644b182d0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b1850510 0x5644b1852960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b182cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/1339396482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b18243b0 0x5644b182c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0caf400640 0 client.25451 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 >> 172.21.15.50:0/1339396482 conn(0x5644b1820920 msgr2=0x5644b1826630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 shutdown_connections 2024-09-12T03:29:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/1339396482 wait complete. 2024-09-12T03:29:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.306+0000 7f0cd3a3c740 20 client.25451 trim_cache size 0 max 16384 2024-09-12T03:29:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.325+0000 7f0cd3a3c740 1 Processor -- start 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.325+0000 7f0cd3a3c740 1 -- start start 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b181b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1858270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a6650 con 0x5644b182f570 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a6d80 con 0x5644b18243b0 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a45f0 con 0x5644b1824cd0 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b181b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b181b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:47578/0 (socket says 172.21.15.50:47578) 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 -- 172.21.15.50:0/3758159768 learned_addr learned my addr 172.21.15.50:0/3758159768 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1858270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 -- 172.21.15.50:0/3758159768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 msgr2=0x5644b1858270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1858270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 -- 172.21.15.50:0/3758159768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 msgr2=0x5644b18587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.326+0000 7f0cc9e00640 1 -- 172.21.15.50:0/3758159768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b1821a90 con 0x5644b18243b0 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.327+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b181b8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005bc0 tx=0x7f0cbc02a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82edbb157fa71fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.327+0000 7f0cba600640 1 -- 172.21.15.50:0/3758159768 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc035050 con 0x5644b18243b0 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.327+0000 7f0cba600640 1 -- 172.21.15.50:0/3758159768 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc0044c0 con 0x5644b18243b0 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cba600640 1 -- 172.21.15.50:0/3758159768 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc003e10 con 0x5644b18243b0 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3758159768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 msgr2=0x5644b181b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b181b8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005bc0 tx=0x7f0cbc02a4d0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3758159768 shutdown_connections 2024-09-12T03:29:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1858270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3758159768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b181b8d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3758159768 >> 172.21.15.50:0/3758159768 conn(0x5644b1820920 msgr2=0x5644b1827930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3758159768 shutdown_connections 2024-09-12T03:29:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3758159768 wait complete. 2024-09-12T03:29:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.328+0000 7f0cd3a3c740 1 -- start start 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 Processor -- start 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1837b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1829290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc074200 con 0x5644b182f570 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a3f20 con 0x5644b18243b0 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a3180 con 0x5644b1824cd0 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.329+0000 7f0cd3a3c740 10 client.0 _ll_register_callbacks cb 0x7f0cc4011e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1829290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1829290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:56592/0 (socket says 172.21.15.50:56592) 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 -- 172.21.15.50:0/3874056431 learned_addr learned my addr 172.21.15.50:0/3874056431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:29:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1837b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cca800640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 -- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 msgr2=0x5644b1837b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1837b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 -- 172.21.15.50:0/3874056431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 msgr2=0x5644b18297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 -- 172.21.15.50:0/3874056431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b18215b0 con 0x5644b1824cd0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cca800640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cc3e00640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1829290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40986e0 tx=0x7f0cb409be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd22ffd1950a3e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0caf400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0caf400640 1 -- 172.21.15.50:0/3874056431 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4018d30 con 0x5644b1824cd0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644b1791b80 con 0x5644b1824cd0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.330+0000 7f0caf400640 1 -- 172.21.15.50:0/3874056431 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f0cb409b160 con 0x5644b1824cd0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0caf400640 1 -- 172.21.15.50:0/3874056431 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cb40b9050 con 0x5644b1824cd0 2024-09-12T03:29:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 10 client.25450 fetch_fsmap learned FSMap version 30 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644b18bafb0 con 0x5644b1824cd0 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0caf400640 1 -- 172.21.15.50:0/3874056431 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644b1791b80 con 0x5644b1824cd0 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0caf400640 1 -- 172.21.15.50:0/3874056431 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0cb4097a60 con 0x5644b1824cd0 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 10 client.25450 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 20 client.25450 populate_metadata read hostname 'smithi050' 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b18bb6f0 0x5644b18bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 4 client.25450 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644b170e7c0 con 0x5644b18bb6f0 2024-09-12T03:29:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.331+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b18bb6f0 0x5644b18bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.332+0000 7f0cc9e00640 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b18bb6f0 0x5644b18bdb10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3301070e4525f40 server_cookie=636209efe5712fc in_seq=0 out_seq=0 2024-09-12T03:29:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.332+0000 7f0caf400640 10 client.25450 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0caf400640 1 -- 172.21.15.50:0/3874056431 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644b170e7c0 con 0x5644b18bb6f0 2024-09-12T03:29:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0caf400640 10 client.25450 handle_command_reply: tid=0 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 client.25450 shutdown 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 msgr2=0x5644b1829290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1829290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40986e0 tx=0x7f0cb409be50 comp rx=0 tx=0).stop 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 shutdown_connections 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644b182f570 0x5644b18297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5644b18bb6f0 0x5644b18bdb10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b1824cd0 0x5644b1829290 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 --2- 172.21.15.50:0/3874056431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5644b18243b0 0x5644b1837b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0caf400640 0 client.25450 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 >> 172.21.15.50:0/3874056431 conn(0x5644b1820920 msgr2=0x5644b1825e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 shutdown_connections 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.348+0000 7f0cd3a3c740 1 -- 172.21.15.50:0/3874056431 wait complete. 2024-09-12T03:29:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.349+0000 7f0cd3a3c740 20 client.25450 trim_cache size 0 max 16384 2024-09-12T03:29:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:29:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:29:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:29:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.349+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0cb00a7100 msgr2=0x7f0cb00a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.349+0000 7f0ccb200640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0cb00a7100 0x7f0cb00a95c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4011890 tx=0x7f0cc4009500 comp rx=0 tx=0).stop 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.349+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 msgr2=0x7f0ccc1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.349+0000 7f0ccb200640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc1964d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9f0 tx=0x7f0cb400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 shutdown_connections 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ccc106960 0x7f0ccc19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ccc100390 0x7f0ccc1964d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0cb00a7100 0x7f0cb00a95c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 --2- 172.21.15.50:0/4117103283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ccc0ff9d0 0x7f0ccc195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 >> 172.21.15.50:0/4117103283 conn(0x7f0ccc0fb820 msgr2=0x7f0ccc0fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 shutdown_connections 2024-09-12T03:29:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:29:57.350+0000 7f0ccb200640 1 -- 172.21.15.50:0/4117103283 wait complete. 2024-09-12T03:29:58.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:29:58 smithi050 ceph-mon[30378]: pgmap v87: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 598 op/s 2024-09-12T03:29:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:29:58 smithi142 ceph-mon[35329]: pgmap v87: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 598 op/s 2024-09-12T03:29:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:29:58 smithi082 ceph-mon[35046]: pgmap v87: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 598 op/s 2024-09-12T03:30:00.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:00 smithi050 ceph-mon[30378]: pgmap v88: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-09-12T03:30:00.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:00 smithi050 ceph-mon[30378]: overall HEALTH_OK 2024-09-12T03:30:00.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:00 smithi142 ceph-mon[35329]: pgmap v88: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-09-12T03:30:00.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:00 smithi142 ceph-mon[35329]: overall HEALTH_OK 2024-09-12T03:30:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:00 smithi082 ceph-mon[35046]: pgmap v88: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-09-12T03:30:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:00 smithi082 ceph-mon[35046]: overall HEALTH_OK 2024-09-12T03:30:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:02 smithi050 ceph-mon[30378]: pgmap v89: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-09-12T03:30:02.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:02 smithi142 ceph-mon[35329]: pgmap v89: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-09-12T03:30:02.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:02 smithi082 ceph-mon[35046]: pgmap v89: 129 pgs: 129 active+clean; 54 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-09-12T03:30:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:04 smithi050 ceph-mon[30378]: pgmap v90: 129 pgs: 129 active+clean; 50 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-09-12T03:30:04.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:04 smithi142 ceph-mon[35329]: pgmap v90: 129 pgs: 129 active+clean; 50 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-09-12T03:30:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:04 smithi082 ceph-mon[35046]: pgmap v90: 129 pgs: 129 active+clean; 50 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 601 op/s 2024-09-12T03:30:06.656 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:06 smithi082 ceph-mon[35046]: pgmap v91: 129 pgs: 129 active+clean; 50 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 392 op/s 2024-09-12T03:30:06.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:06 smithi050 ceph-mon[30378]: pgmap v91: 129 pgs: 129 active+clean; 50 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 392 op/s 2024-09-12T03:30:06.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:06 smithi142 ceph-mon[35329]: pgmap v91: 129 pgs: 129 active+clean; 50 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 392 op/s 2024-09-12T03:30:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:08 smithi050 ceph-mon[30378]: pgmap v92: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 563 op/s 2024-09-12T03:30:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:08 smithi142 ceph-mon[35329]: pgmap v92: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 563 op/s 2024-09-12T03:30:08.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:08 smithi082 ceph-mon[35046]: pgmap v92: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 563 op/s 2024-09-12T03:30:09.270 INFO:tasks.workunit.client.0.smithi050.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-09-12T03:30:09.799 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xffcdd 2024-09-12T03:30:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 -- 172.21.15.50:0/4012255670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820103f30 msgr2=0x7fd820104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 --2- 172.21.15.50:0/4012255670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820103f30 0x7fd820104310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd810009900 tx=0x7fd81002a400 comp rx=0 tx=0).stop 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 -- 172.21.15.50:0/4012255670 shutdown_connections 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 --2- 172.21.15.50:0/4012255670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820109080 0x7fd820110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 --2- 172.21.15.50:0/4012255670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8201048e0 0x7fd820108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 --2- 172.21.15.50:0/4012255670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820103f30 0x7fd820104310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 -- 172.21.15.50:0/4012255670 >> 172.21.15.50:0/4012255670 conn(0x7fd8200ff980 msgr2=0x7fd820101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 -- 172.21.15.50:0/4012255670 shutdown_connections 2024-09-12T03:30:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.487+0000 7fd825c00640 1 -- 172.21.15.50:0/4012255670 wait complete. 2024-09-12T03:30:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.488+0000 7fd825c00640 1 Processor -- start 2024-09-12T03:30:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.488+0000 7fd825c00640 1 -- start start 2024-09-12T03:30:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd825c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 0x7fd82019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd825c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820109080 0x7fd82019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd825c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd82019ee50 0x7fd82019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd825c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd820114120 con 0x7fd82019ee50 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd825c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd820113fa0 con 0x7fd820109080 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd825c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201142a0 con 0x7fd820103f30 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 0x7fd82019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 0x7fd82019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50818/0 (socket says 172.21.15.50:50818) 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 -- 172.21.15.50:0/508476086 learned_addr learned my addr 172.21.15.50:0/508476086 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd817e00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820109080 0x7fd82019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81f400640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd82019ee50 0x7fd82019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 -- 172.21.15.50:0/508476086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820109080 msgr2=0x7fd82019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820109080 0x7fd82019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 -- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd82019ee50 msgr2=0x7fd82019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd82019ee50 0x7fd82019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81ea00640 1 -- 172.21.15.50:0/508476086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82019f940 con 0x7fd820103f30 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.489+0000 7fd81f400640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd82019ee50 0x7fd82019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.490+0000 7fd81ea00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 0x7fd82019a590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd810004770 tx=0x7fd810003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.490+0000 7fd817400640 1 -- 172.21.15.50:0/508476086 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81003f070 con 0x7fd820103f30 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.490+0000 7fd817400640 1 -- 172.21.15.50:0/508476086 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd810031ca0 con 0x7fd820103f30 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.490+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82019fbd0 con 0x7fd820103f30 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.490+0000 7fd817400640 1 -- 172.21.15.50:0/508476086 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810035030 con 0x7fd820103f30 2024-09-12T03:30:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.490+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8201a73f0 con 0x7fd820103f30 2024-09-12T03:30:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.493+0000 7fd828bd7740 1 Processor -- start 2024-09-12T03:30:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd817400640 1 -- 172.21.15.50:0/508476086 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd810042070 con 0x7fd820103f30 2024-09-12T03:30:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 -- start start 2024-09-12T03:30:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x5585805f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f4b00 0x5585805fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580601790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55858055a100 con 0x5585805ff3a0 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5585805021b0 con 0x5585805f41e0 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd828bd7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585805560c0 con 0x5585805f4b00 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd81f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x5585805f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd81f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x5585805f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41262/0 (socket says 172.21.15.50:41262) 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd81f400640 1 -- 172.21.15.50:0/2889872614 learned_addr learned my addr 172.21.15.50:0/2889872614 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.494+0000 7fd81ea00640 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f4b00 0x5585805fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd817e00640 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580601790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd81f400640 1 -- 172.21.15.50:0/2889872614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f4b00 msgr2=0x5585805fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd81f400640 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f4b00 0x5585805fee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd81f400640 1 -- 172.21.15.50:0/2889872614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 msgr2=0x558580601790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd81f400640 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580601790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd81f400640 1 -- 172.21.15.50:0/2889872614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858026d210 con 0x5585805f41e0 2024-09-12T03:30:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd817e00640 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580601790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd81f400640 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x5585805f45c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd818009fd0 tx=0x7fd818010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5187d52e135b8386 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd80fe00640 1 -- 172.21.15.50:0/2889872614 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818011700 con 0x5585805f41e0 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd80fe00640 1 -- 172.21.15.50:0/2889872614 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd818011d80 con 0x5585805f41e0 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.495+0000 7fd80fe00640 1 -- 172.21.15.50:0/2889872614 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81800e3c0 con 0x5585805f41e0 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd817400640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd8040a2e20 0x7fd8040a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 -- 172.21.15.50:0/2889872614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 msgr2=0x5585805f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x5585805f45c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd818009fd0 tx=0x7fd818010860 comp rx=0 tx=0).stop 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd817400640 1 -- 172.21.15.50:0/508476086 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fd8100efa80 con 0x7fd820103f30 2024-09-12T03:30:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd81ea00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd8040a2e20 0x7fd8040a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 -- 172.21.15.50:0/2889872614 shutdown_connections 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580601790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f4b00 0x5585805fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 --2- 172.21.15.50:0/2889872614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x5585805f45c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.496+0000 7fd828bd7740 1 -- 172.21.15.50:0/2889872614 >> 172.21.15.50:0/2889872614 conn(0x5585805f0750 msgr2=0x5585805f0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.497+0000 7fd828bd7740 1 -- 172.21.15.50:0/2889872614 shutdown_connections 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.497+0000 7fd828bd7740 1 -- 172.21.15.50:0/2889872614 wait complete. 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.497+0000 7fd828bd7740 1 -- start start 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 Processor -- start 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd81ea00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd8040a2e20 0x7fd8040a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ecef0 tx=0x7fd8100f35c0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x5585805fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x5585805fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805ff3a0 0x5585805fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558580601f70 con 0x5585805f4b00 2024-09-12T03:30:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5585804e3090 con 0x5585805ff3a0 2024-09-12T03:30:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585806020f0 con 0x5585805f41e0 2024-09-12T03:30:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.498+0000 7fd828bd7740 10 client.0 _ll_register_callbacks cb 0x558580537f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x5585805fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x5585805fc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50844/0 (socket says 172.21.15.50:50844) 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 -- 172.21.15.50:0/3300272255 learned_addr learned my addr 172.21.15.50:0/3300272255 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd817e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x5585805fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805ff3a0 0x5585805fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 -- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805ff3a0 msgr2=0x5585805fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805ff3a0 0x5585805fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 -- 172.21.15.50:0/3300272255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 msgr2=0x5585805fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x5585805fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.499+0000 7fd81f400640 1 -- 172.21.15.50:0/3300272255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558580339620 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd81f400640 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x5585805fc3a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd818005cc0 tx=0x7fd81801bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709f78a428d6872c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd80cc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd80cc00640 1 -- 172.21.15.50:0/3300272255 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818011910 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd80cc00640 1 -- 172.21.15.50:0/3300272255 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fd818011ab0 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd818011d80 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd80cc00640 1 -- 172.21.15.50:0/3300272255 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd81803b050 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd828bd7740 10 client.25462 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55858060db90 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.500+0000 7fd80cc00640 1 -- 172.21.15.50:0/3300272255 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd818011d80 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd80cc00640 1 -- 172.21.15.50:0/3300272255 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd8180363d0 con 0x5585805f41e0 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd828bd7740 10 client.25462 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd828bd7740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd828bd7740 20 client.25462 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd828bd7740 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55858060efa0 0x558580611390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd828bd7740 4 client.25462 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558580567830 con 0x55858060efa0 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.501+0000 7fd817e00640 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55858060efa0 0x558580611390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.502+0000 7fd817e00640 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55858060efa0 0x558580611390 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3279710dbbf58f1f server_cookie=8882d36c6803fa6f in_seq=0 out_seq=0 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.502+0000 7fd80cc00640 10 client.25462 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.502+0000 7fd80cc00640 1 -- 172.21.15.50:0/3300272255 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558580567830 con 0x55858060efa0 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.502+0000 7fd80cc00640 10 client.25462 handle_command_reply: tid=0 2024-09-12T03:30:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.502+0000 7fd828bd7740 1 client.25462 shutdown 2024-09-12T03:30:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 msgr2=0x5585805fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x5585805fc3a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd818005cc0 tx=0x7fd81801bc30 comp rx=0 tx=0).stop 2024-09-12T03:30:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 shutdown_connections 2024-09-12T03:30:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805ff3a0 0x5585805fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x5585805fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55858060efa0 0x558580611390 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 --2- 172.21.15.50:0/3300272255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x5585805fc3a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd80cc00640 0 client.25462 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 >> 172.21.15.50:0/3300272255 conn(0x5585805f0750 msgr2=0x5585805f6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 shutdown_connections 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 1 -- 172.21.15.50:0/3300272255 wait complete. 2024-09-12T03:30:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.503+0000 7fd828bd7740 20 client.25462 trim_cache size 0 max 16384 2024-09-12T03:30:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 Processor -- start 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 -- start start 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x558580616a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x558580616f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805ff3a0 0x5585805f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201af6a0 con 0x5585805f4b00 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201acf10 con 0x5585805f41e0 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd828bd7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201aad60 con 0x5585805ff3a0 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd81f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x558580616a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd817e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x558580616f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd81f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x558580616a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41278/0 (socket says 172.21.15.50:41278) 2024-09-12T03:30:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd81f400640 1 -- 172.21.15.50:0/1129063764 learned_addr learned my addr 172.21.15.50:0/1129063764 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.521+0000 7fd81f400640 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805ff3a0 0x5585805f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd81f400640 1 -- 172.21.15.50:0/1129063764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805ff3a0 msgr2=0x5585805f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd81f400640 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805ff3a0 0x5585805f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd81f400640 1 -- 172.21.15.50:0/1129063764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 msgr2=0x558580616f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd81f400640 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x558580616f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd81f400640 1 -- 172.21.15.50:0/1129063764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585805eac20 con 0x5585805f41e0 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd817e00640 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x558580616f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd81f400640 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x558580616a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd818005d70 tx=0x7fd818009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c424227732dbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd80f400640 1 -- 172.21.15.50:0/1129063764 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818009680 con 0x5585805f41e0 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd80f400640 1 -- 172.21.15.50:0/1129063764 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd81801b920 con 0x5585805f41e0 2024-09-12T03:30:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.522+0000 7fd80f400640 1 -- 172.21.15.50:0/1129063764 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818011a80 con 0x5585805f41e0 2024-09-12T03:30:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 -- 172.21.15.50:0/1129063764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 msgr2=0x558580616a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x558580616a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd818005d70 tx=0x7fd818009610 comp rx=0 tx=0).stop 2024-09-12T03:30:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 -- 172.21.15.50:0/1129063764 shutdown_connections 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805ff3a0 0x5585805f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805f4b00 0x558580616f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 --2- 172.21.15.50:0/1129063764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f41e0 0x558580616a20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 -- 172.21.15.50:0/1129063764 >> 172.21.15.50:0/1129063764 conn(0x5585805f0750 msgr2=0x5585805fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 -- 172.21.15.50:0/1129063764 shutdown_connections 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 -- 172.21.15.50:0/1129063764 wait complete. 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.523+0000 7fd828bd7740 1 -- start start 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 Processor -- start 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x558580607810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 0x558580602b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580603070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201ab3d0 con 0x5585805ff3a0 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201ac840 con 0x5585805f4b00 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201abaa0 con 0x5585805f41e0 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd828bd7740 10 client.0 _ll_register_callbacks cb 0x7fd8100f3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd817e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 0x558580602b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd817e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 0x558580602b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41286/0 (socket says 172.21.15.50:41286) 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd817e00640 1 -- 172.21.15.50:0/4003550371 learned_addr learned my addr 172.21.15.50:0/4003550371 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.524+0000 7fd81f400640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x558580607810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd81f400640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580603070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd817e00640 1 -- 172.21.15.50:0/4003550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 msgr2=0x558580607810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd817e00640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x558580607810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd817e00640 1 -- 172.21.15.50:0/4003550371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 msgr2=0x558580603070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd817e00640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580603070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd817e00640 1 -- 172.21.15.50:0/4003550371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558580603630 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd817e00640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 0x558580602b30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5585805f6d80 tx=0x7fd808014900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff10d50b57e64f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd80cc00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd80cc00640 1 -- 172.21.15.50:0/4003550371 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80801e610 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd81801b920 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd80cc00640 1 -- 172.21.15.50:0/4003550371 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fd80801e7b0 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.525+0000 7fd80cc00640 1 -- 172.21.15.50:0/4003550371 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd80802f050 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 10 client.25475 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585806797f0 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd80cc00640 1 -- 172.21.15.50:0/4003550371 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd81801b920 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd80cc00640 1 -- 172.21.15.50:0/4003550371 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd808025030 con 0x5585805f4b00 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 10 client.25475 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 20 client.25475 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558580679fb0 0x55858067c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 4 client.25475 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558580561b80 con 0x558580679fb0 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.526+0000 7fd81f400640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558580679fb0 0x55858067c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.527+0000 7fd81f400640 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558580679fb0 0x55858067c3d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba2c4dbe7cb323f server_cookie=98bef4f9de91267d in_seq=0 out_seq=0 2024-09-12T03:30:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.527+0000 7fd80cc00640 10 client.25475 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd80cc00640 1 -- 172.21.15.50:0/4003550371 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558580561b80 con 0x558580679fb0 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd80cc00640 10 client.25475 handle_command_reply: tid=0 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 client.25475 shutdown 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 msgr2=0x558580602b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 0x558580602b30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5585805f6d80 tx=0x7fd808014900 comp rx=0 tx=0).stop 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 shutdown_connections 2024-09-12T03:30:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585805ff3a0 0x558580603070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585805f4b00 0x558580602b30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558580679fb0 0x55858067c3d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 --2- 172.21.15.50:0/4003550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585805f41e0 0x558580607810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd80cc00640 0 client.25475 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 >> 172.21.15.50:0/4003550371 conn(0x5585805f0750 msgr2=0x5585805f5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 shutdown_connections 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 1 -- 172.21.15.50:0/4003550371 wait complete. 2024-09-12T03:30:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.543+0000 7fd828bd7740 20 client.25475 trim_cache size 0 max 16384 2024-09-12T03:30:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:30:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:30:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:30:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:30:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd8040a2e20 msgr2=0x7fd8040a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd8040a2e20 0x7fd8040a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ecef0 tx=0x7fd8100f35c0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 msgr2=0x7fd82019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 0x7fd82019a590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd810004770 tx=0x7fd810003c10 comp rx=0 tx=0).stop 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 shutdown_connections 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd8040a2e20 0x7fd8040a52e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd82019ee50 0x7fd82019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd820109080 0x7fd82019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 --2- 172.21.15.50:0/508476086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd820103f30 0x7fd82019a590 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 >> 172.21.15.50:0/508476086 conn(0x7fd8200ff980 msgr2=0x7fd820100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.545+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 shutdown_connections 2024-09-12T03:30:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:10.546+0000 7fd825c00640 1 -- 172.21.15.50:0/508476086 wait complete. 2024-09-12T03:30:10.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:10 smithi050 ceph-mon[30378]: pgmap v93: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2024-09-12T03:30:10.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:10 smithi142 ceph-mon[35329]: pgmap v93: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2024-09-12T03:30:10.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:10 smithi082 ceph-mon[35046]: pgmap v93: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2024-09-12T03:30:12.357 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xffd97 2024-09-12T03:30:12.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:12 smithi050 ceph-mon[30378]: pgmap v94: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2024-09-12T03:30:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:12 smithi142 ceph-mon[35329]: pgmap v94: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2024-09-12T03:30:12.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:12 smithi082 ceph-mon[35046]: pgmap v94: 129 pgs: 129 active+clean; 54 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2024-09-12T03:30:14.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:14 smithi050 ceph-mon[30378]: pgmap v95: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 532 op/s 2024-09-12T03:30:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:14 smithi142 ceph-mon[35329]: pgmap v95: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 532 op/s 2024-09-12T03:30:14.853 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:14 smithi082 ceph-mon[35046]: pgmap v95: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 532 op/s 2024-09-12T03:30:16.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:16 smithi050 ceph-mon[30378]: pgmap v96: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 334 op/s 2024-09-12T03:30:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:16 smithi142 ceph-mon[35329]: pgmap v96: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 334 op/s 2024-09-12T03:30:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:16 smithi082 ceph-mon[35046]: pgmap v96: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 334 op/s 2024-09-12T03:30:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:18 smithi050 ceph-mon[30378]: pgmap v97: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 484 op/s 2024-09-12T03:30:18.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:18 smithi142 ceph-mon[35329]: pgmap v97: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 484 op/s 2024-09-12T03:30:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:18 smithi082 ceph-mon[35046]: pgmap v97: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 484 op/s 2024-09-12T03:30:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:20 smithi050 ceph-mon[30378]: pgmap v98: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-12T03:30:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:20 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:20 smithi142 ceph-mon[35329]: pgmap v98: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-12T03:30:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:20 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:20 smithi082 ceph-mon[35046]: pgmap v98: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-12T03:30:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:20 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:21.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:21 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:21.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:21 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:21.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:21 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:30:21.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:21 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:21.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:21 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:21.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:21 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:30:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:21 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:21 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:21.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:21 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:30:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:22 smithi142 ceph-mon[35329]: pgmap v99: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-12T03:30:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:22 smithi082 ceph-mon[35046]: pgmap v99: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-12T03:30:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:22 smithi050 ceph-mon[30378]: pgmap v99: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-12T03:30:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 -- 172.21.15.50:0/339389833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8074f40 msgr2=0x7fa7a810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 --2- 172.21.15.50:0/339389833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8074f40 0x7fa7a810dde0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa798009900 tx=0x7fa79802a480 comp rx=0 tx=0).stop 2024-09-12T03:30:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 -- 172.21.15.50:0/339389833 shutdown_connections 2024-09-12T03:30:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 --2- 172.21.15.50:0/339389833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a810e320 0x7fa7a8110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 --2- 172.21.15.50:0/339389833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8074f40 0x7fa7a810dde0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 --2- 172.21.15.50:0/339389833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7a8074620 0x7fa7a8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 -- 172.21.15.50:0/339389833 >> 172.21.15.50:0/339389833 conn(0x7fa7a80fd7e0 msgr2=0x7fa7a80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 -- 172.21.15.50:0/339389833 shutdown_connections 2024-09-12T03:30:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.687+0000 7fa7b0800640 1 -- 172.21.15.50:0/339389833 wait complete. 2024-09-12T03:30:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.688+0000 7fa7b0800640 1 Processor -- start 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 -- start start 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 0x7fa7a819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7a8074f40 0x7fa7a819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a810e320 0x7fa7a819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8114160 con 0x7fa7a8074f40 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8113fe0 con 0x7fa7a810e320 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7b0800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81142e0 con 0x7fa7a8074620 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 0x7fa7a819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 0x7fa7a819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:48018/0 (socket says 172.21.15.50:48018) 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7ade00640 1 -- 172.21.15.50:0/3393134956 learned_addr learned my addr 172.21.15.50:0/3393134956 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7ae800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a810e320 0x7fa7a819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.689+0000 7fa7ad400640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7a8074f40 0x7fa7a819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7ade00640 1 -- 172.21.15.50:0/3393134956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a810e320 msgr2=0x7fa7a819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7ade00640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a810e320 0x7fa7a819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7ade00640 1 -- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7a8074f40 msgr2=0x7fa7a819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7ade00640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7a8074f40 0x7fa7a819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7ade00640 1 -- 172.21.15.50:0/3393134956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a819f5a0 con 0x7fa7a8074620 2024-09-12T03:30:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7ade00640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 0x7fa7a819a5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00e9f0 tx=0x7fa79c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7a2a00640 1 -- 172.21.15.50:0/3393134956 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00cd80 con 0x7fa7a8074620 2024-09-12T03:30:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7a2a00640 1 -- 172.21.15.50:0/3393134956 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c010910 con 0x7fa7a8074620 2024-09-12T03:30:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a819f890 con 0x7fa7a8074620 2024-09-12T03:30:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.690+0000 7fa7a2a00640 1 -- 172.21.15.50:0/3393134956 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c018990 con 0x7fa7a8074620 2024-09-12T03:30:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.691+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81a7120 con 0x7fa7a8074620 2024-09-12T03:30:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.694+0000 7fa7a2a00640 1 -- 172.21.15.50:0/3393134956 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa79c010ac0 con 0x7fa7a8074620 2024-09-12T03:30:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 Processor -- start 2024-09-12T03:30:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 -- start start 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7a2a00640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa7900a2da0 0x7fa7900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7a2a00640 1 -- 172.21.15.50:0/3393134956 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fa79c0cfb30 con 0x7fa7a8074620 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f45b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f465e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f468790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564e8f3c1100 con 0x564e8f45bb00 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564e8f3691b0 con 0x564e8f45b1e0 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564e8f3bd0c0 con 0x564e8f4663a0 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7ae800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa7900a2da0 0x7fa7900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f45b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f45b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:50844/0 (socket says 172.21.15.50:50844) 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.696+0000 7fa7ade00640 1 -- 172.21.15.50:0/3004630307 learned_addr learned my addr 172.21.15.50:0/3004630307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ae800640 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f468790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ad400640 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f465e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ade00640 1 -- 172.21.15.50:0/3004630307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 msgr2=0x564e8f468790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ade00640 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f468790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ade00640 1 -- 172.21.15.50:0/3004630307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 msgr2=0x564e8f465e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ade00640 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f465e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ade00640 1 -- 172.21.15.50:0/3004630307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8f0d4210 con 0x564e8f45b1e0 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ad400640 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f465e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ade00640 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f45b5c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0a43a0 tx=0x7fa79c09bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad6f85c233d7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa797000640 1 -- 172.21.15.50:0/3004630307 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0cd110 con 0x564e8f45b1e0 2024-09-12T03:30:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa797000640 1 -- 172.21.15.50:0/3004630307 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c0b1030 con 0x564e8f45b1e0 2024-09-12T03:30:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa7ae800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa7900a2da0 0x7fa7900a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009900 tx=0x7fa7a4009ea0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.697+0000 7fa797000640 1 -- 172.21.15.50:0/3004630307 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c09b160 con 0x564e8f45b1e0 2024-09-12T03:30:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.698+0000 7fa7b31fd740 1 -- 172.21.15.50:0/3004630307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 msgr2=0x564e8f45b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.699+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f45b5c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0a43a0 tx=0x7fa79c09bc10 comp rx=0 tx=0).stop 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 -- 172.21.15.50:0/3004630307 shutdown_connections 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f468790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f465e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/3004630307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f45b5c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 -- 172.21.15.50:0/3004630307 >> 172.21.15.50:0/3004630307 conn(0x564e8f457750 msgr2=0x564e8f457b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 -- 172.21.15.50:0/3004630307 shutdown_connections 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.700+0000 7fa7b31fd740 1 -- 172.21.15.50:0/3004630307 wait complete. 2024-09-12T03:30:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.701+0000 7fa7b31fd740 1 -- start start 2024-09-12T03:30:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.701+0000 7fa7b31fd740 1 Processor -- start 2024-09-12T03:30:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.701+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f4632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.701+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f463800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f463d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564e8f468f70 con 0x564e8f45bb00 2024-09-12T03:30:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564e8f34a090 con 0x564e8f45b1e0 2024-09-12T03:30:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564e8f4690f0 con 0x564e8f4663a0 2024-09-12T03:30:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7b31fd740 10 client.0 _ll_register_callbacks cb 0x564e8f39ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f4632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f4632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:50846/0 (socket says 172.21.15.50:50846) 2024-09-12T03:30:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 -- 172.21.15.50:0/4002086537 learned_addr learned my addr 172.21.15.50:0/4002086537 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ad400640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f463800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ae800640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f463d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 -- 172.21.15.50:0/4002086537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 msgr2=0x564e8f463d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f463d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 -- 172.21.15.50:0/4002086537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 msgr2=0x564e8f463800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f463800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 -- 172.21.15.50:0/4002086537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e8f1a0620 con 0x564e8f45b1e0 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ad400640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f463800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa7ade00640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f4632c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa79c098170 tx=0x7fa79c018bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f81a407b26b4d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.702+0000 7fa78b400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa78b400640 1 -- 172.21.15.50:0/4002086537 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0a9060 con 0x564e8f45b1e0 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa78b400640 1 -- 172.21.15.50:0/4002086537 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fa79c09b360 con 0x564e8f45b1e0 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79c0b1030 con 0x564e8f45b1e0 2024-09-12T03:30:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa78b400640 1 -- 172.21.15.50:0/4002086537 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa79c0ba070 con 0x564e8f45b1e0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 10 client.25487 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564e8f474aa0 con 0x564e8f45b1e0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa78b400640 1 -- 172.21.15.50:0/4002086537 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c0b1030 con 0x564e8f45b1e0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa78b400640 1 -- 172.21.15.50:0/4002086537 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa79c0a11a0 con 0x564e8f45b1e0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 10 client.25487 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 10 client.25487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 20 client.25487 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f475df0 0x564e8f4781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 4 client.25487 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.703+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e8f3ce830 con 0x564e8f475df0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.704+0000 7fa7ad400640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f475df0 0x564e8f4781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.704+0000 7fa7ad400640 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f475df0 0x564e8f4781e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21135cac7833186 server_cookie=2b18e3e531d0beb3 in_seq=0 out_seq=0 2024-09-12T03:30:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.704+0000 7fa78b400640 10 client.25487 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa78b400640 1 -- 172.21.15.50:0/4002086537 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e8f3ce830 con 0x564e8f475df0 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa78b400640 10 client.25487 handle_command_reply: tid=0 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 client.25487 shutdown 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 msgr2=0x564e8f4632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f4632c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa79c098170 tx=0x7fa79c018bb0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 shutdown_connections 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f475df0 0x564e8f4781e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f4663a0 0x564e8f463d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f45bb00 0x564e8f463800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa78b400640 0 client.25487 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4002086537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f4632c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.705+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 >> 172.21.15.50:0/4002086537 conn(0x564e8f457750 msgr2=0x564e8f45d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.706+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 shutdown_connections 2024-09-12T03:30:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.706+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4002086537 wait complete. 2024-09-12T03:30:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.706+0000 7fa7b31fd740 20 client.25487 trim_cache size 0 max 16384 2024-09-12T03:30:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.723+0000 7fa7b31fd740 1 Processor -- start 2024-09-12T03:30:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 -- start start 2024-09-12T03:30:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f465a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f462b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f471340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b3b60 con 0x564e8f4663a0 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b13d0 con 0x564e8f45b1e0 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81af220 con 0x564e8f45bb00 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f465a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7ae800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f471340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f465a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:50856/0 (socket says 172.21.15.50:50856) 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7ad400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f462b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.724+0000 7fa7ade00640 1 -- 172.21.15.50:0/4194148338 learned_addr learned my addr 172.21.15.50:0/4194148338 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ade00640 1 -- 172.21.15.50:0/4194148338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 msgr2=0x564e8f462b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ade00640 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f462b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ade00640 1 -- 172.21.15.50:0/4194148338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 msgr2=0x564e8f471340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ade00640 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f471340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ade00640 1 -- 172.21.15.50:0/4194148338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8f457580 con 0x564e8f45b1e0 2024-09-12T03:30:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ad400640 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f462b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa7ade00640 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f465a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00c6a0 tx=0x7fa79c09bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2349157a9b8d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa796600640 1 -- 172.21.15.50:0/4194148338 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0a9070 con 0x564e8f45b1e0 2024-09-12T03:30:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa796600640 1 -- 172.21.15.50:0/4194148338 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c0b1020 con 0x564e8f45b1e0 2024-09-12T03:30:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.725+0000 7fa796600640 1 -- 172.21.15.50:0/4194148338 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0b4e60 con 0x564e8f45b1e0 2024-09-12T03:30:23.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4194148338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 msgr2=0x564e8f465a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f465a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00c6a0 tx=0x7fa79c09bc10 comp rx=0 tx=0).stop 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4194148338 shutdown_connections 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f471340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f462b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/4194148338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f465a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4194148338 >> 172.21.15.50:0/4194148338 conn(0x564e8f457750 msgr2=0x564e8f45d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4194148338 shutdown_connections 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.726+0000 7fa7b31fd740 1 -- 172.21.15.50:0/4194148338 wait complete. 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 -- start start 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 Processor -- start 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f47b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f478c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f4791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81af890 con 0x564e8f4663a0 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b0d00 con 0x564e8f45b1e0 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81aff60 con 0x564e8f45bb00 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.727+0000 7fa7b31fd740 10 client.0 _ll_register_callbacks cb 0x7fa79c09b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f47b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ad400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f478c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f47b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:50862/0 (socket says 172.21.15.50:50862) 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 -- 172.21.15.50:0/479770492 learned_addr learned my addr 172.21.15.50:0/479770492 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ae800640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f4791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 -- 172.21.15.50:0/479770492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 msgr2=0x564e8f478c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f478c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 -- 172.21.15.50:0/479770492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 msgr2=0x564e8f4791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f4791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 -- 172.21.15.50:0/479770492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e8f4583e0 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ad400640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f478c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa7ade00640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f47b950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa79c09ba10 tx=0x7fa79c09cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da9e4a49b1c8c821 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa78b400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa78b400640 1 -- 172.21.15.50:0/479770492 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0a9070 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.728+0000 7fa78b400640 1 -- 172.21.15.50:0/479770492 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fa79c086600 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79c0b1020 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa78b400640 1 -- 172.21.15.50:0/479770492 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa79c0b4e60 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 10 client.25493 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564e8f458800 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa78b400640 1 -- 172.21.15.50:0/479770492 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c0b1020 con 0x564e8f45b1e0 2024-09-12T03:30:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa78b400640 1 -- 172.21.15.50:0/479770492 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa79c010430 con 0x564e8f45b1e0 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 10 client.25493 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 20 client.25493 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f4e06d0 0x564e8f4e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 4 client.25493 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e8f3c8b80 con 0x564e8f4e06d0 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.729+0000 7fa7ad400640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f4e06d0 0x564e8f4e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.730+0000 7fa78b400640 10 client.25493 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.730+0000 7fa7ad400640 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f4e06d0 0x564e8f4e2ac0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931b9ca32725128e server_cookie=8b3238a66f83edf4 in_seq=0 out_seq=0 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.743+0000 7fa78b400640 1 -- 172.21.15.50:0/479770492 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e8f3c8b80 con 0x564e8f4e06d0 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.743+0000 7fa78b400640 10 client.25493 handle_command_reply: tid=0 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 client.25493 shutdown 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 msgr2=0x564e8f47b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f47b950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa79c09ba10 tx=0x7fa79c09cf90 comp rx=0 tx=0).stop 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 shutdown_connections 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e8f4663a0 0x564e8f4791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x564e8f4e06d0 0x564e8f4e2ac0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e8f45bb00 0x564e8f478c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 --2- 172.21.15.50:0/479770492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564e8f45b1e0 0x564e8f47b950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa78b400640 0 client.25493 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 >> 172.21.15.50:0/479770492 conn(0x564e8f457750 msgr2=0x564e8f45cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 shutdown_connections 2024-09-12T03:30:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.744+0000 7fa7b31fd740 1 -- 172.21.15.50:0/479770492 wait complete. 2024-09-12T03:30:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.745+0000 7fa7b31fd740 20 client.25493 trim_cache size 0 max 16384 2024-09-12T03:30:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:30:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:30:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:30:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.745+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa7900a2da0 msgr2=0x7fa7900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.745+0000 7fa7b0800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa7900a2da0 0x7fa7900a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009900 tx=0x7fa7a4009ea0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.745+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 msgr2=0x7fa7a819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.745+0000 7fa7b0800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 0x7fa7a819a5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00e9f0 tx=0x7fa79c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 shutdown_connections 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a810e320 0x7fa7a819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa7900a2da0 0x7fa7900a5260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7a8074f40 0x7fa7a819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 --2- 172.21.15.50:0/3393134956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7a8074620 0x7fa7a819a5b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 >> 172.21.15.50:0/3393134956 conn(0x7fa7a80fd7e0 msgr2=0x7fa7a80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 shutdown_connections 2024-09-12T03:30:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:23.746+0000 7fa7b0800640 1 -- 172.21.15.50:0/3393134956 wait complete. 2024-09-12T03:30:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:24 smithi142 ceph-mon[35329]: pgmap v100: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 449 op/s 2024-09-12T03:30:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:24 smithi082 ceph-mon[35046]: pgmap v100: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 449 op/s 2024-09-12T03:30:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:24 smithi050 ceph-mon[30378]: pgmap v100: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 449 op/s 2024-09-12T03:30:25.525 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0xffdb8 2024-09-12T03:30:27.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:26 smithi142 ceph-mon[35329]: pgmap v101: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-09-12T03:30:27.872 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:26 smithi050 ceph-mon[30378]: pgmap v101: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-09-12T03:30:27.872 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:26 smithi082 ceph-mon[35046]: pgmap v101: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-09-12T03:30:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:28 smithi142 ceph-mon[35329]: pgmap v102: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 422 op/s 2024-09-12T03:30:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:28 smithi082 ceph-mon[35046]: pgmap v102: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 422 op/s 2024-09-12T03:30:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:28 smithi050 ceph-mon[30378]: pgmap v102: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 422 op/s 2024-09-12T03:30:31.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:30 smithi142 ceph-mon[35329]: pgmap v103: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-12T03:30:31.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:30 smithi082 ceph-mon[35046]: pgmap v103: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-12T03:30:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:30 smithi050 ceph-mon[30378]: pgmap v103: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-12T03:30:33.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:32 smithi142 ceph-mon[35329]: pgmap v104: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-12T03:30:33.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:32 smithi082 ceph-mon[35046]: pgmap v104: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-12T03:30:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:32 smithi050 ceph-mon[30378]: pgmap v104: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-12T03:30:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:34 smithi142 ceph-mon[35329]: pgmap v105: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 403 op/s 2024-09-12T03:30:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:34 smithi082 ceph-mon[35046]: pgmap v105: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 403 op/s 2024-09-12T03:30:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:34 smithi050 ceph-mon[30378]: pgmap v105: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 403 op/s 2024-09-12T03:30:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 -- 172.21.15.50:0/1682729993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584104900 msgr2=0x7fa584108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 --2- 172.21.15.50:0/1682729993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584104900 0x7fa584108940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa570009860 tx=0x7fa57002a340 comp rx=0 tx=0).stop 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 -- 172.21.15.50:0/1682729993 shutdown_connections 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 --2- 172.21.15.50:0/1682729993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5841090a0 0x7fa584110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 --2- 172.21.15.50:0/1682729993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584104900 0x7fa584108940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 --2- 172.21.15.50:0/1682729993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa584103f50 0x7fa584104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.884+0000 7fa58be00640 1 -- 172.21.15.50:0/1682729993 >> 172.21.15.50:0/1682729993 conn(0x7fa5840ff9e0 msgr2=0x7fa584101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.885+0000 7fa58be00640 1 -- 172.21.15.50:0/1682729993 shutdown_connections 2024-09-12T03:30:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.885+0000 7fa58be00640 1 -- 172.21.15.50:0/1682729993 wait complete. 2024-09-12T03:30:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 Processor -- start 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 -- start start 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 0x7fa58419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5841090a0 0x7fa58419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58419ee90 0x7fa58419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841140a0 con 0x7fa5841090a0 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa584113f20 con 0x7fa584103f50 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa58be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa584114220 con 0x7fa58419ee90 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa589400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 0x7fa58419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa589400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 0x7fa58419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41082/0 (socket says 172.21.15.50:41082) 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa589400640 1 -- 172.21.15.50:0/464613389 learned_addr learned my addr 172.21.15.50:0/464613389 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.886+0000 7fa589e00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58419ee90 0x7fa58419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa588a00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5841090a0 0x7fa58419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa589400640 1 -- 172.21.15.50:0/464613389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58419ee90 msgr2=0x7fa58419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa589400640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58419ee90 0x7fa58419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa589400640 1 -- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5841090a0 msgr2=0x7fa58419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa589400640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5841090a0 0x7fa58419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa589400640 1 -- 172.21.15.50:0/464613389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58419f980 con 0x7fa584103f50 2024-09-12T03:30:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa589400640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 0x7fa58419a5d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9c0 tx=0x7fa57800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa582000640 1 -- 172.21.15.50:0/464613389 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57800cde0 con 0x7fa584103f50 2024-09-12T03:30:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa582000640 1 -- 172.21.15.50:0/464613389 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5780035f0 con 0x7fa584103f50 2024-09-12T03:30:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.887+0000 7fa582000640 1 -- 172.21.15.50:0/464613389 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578010690 con 0x7fa584103f50 2024-09-12T03:30:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.888+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58419fc70 con 0x7fa584103f50 2024-09-12T03:30:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.888+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5841a73e0 con 0x7fa584103f50 2024-09-12T03:30:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.891+0000 7fa58ecc1740 1 Processor -- start 2024-09-12T03:30:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.891+0000 7fa58ecc1740 1 -- start start 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfcffe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffeb00 0x555bfd008e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd00b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555bfcf64100 con 0x555bfcffeb00 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555bfcf0c1b0 con 0x555bfd0093a0 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555bfcf600c0 con 0x555bfcffe1e0 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfcffe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfcffe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53276/0 (socket says 172.21.15.50:53276) 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 -- 172.21.15.50:0/2920673946 learned_addr learned my addr 172.21.15.50:0/2920673946 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa582000640 1 -- 172.21.15.50:0/464613389 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa5780037a0 con 0x7fa584103f50 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa588a00640 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd00b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 -- 172.21.15.50:0/2920673946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 msgr2=0x555bfd00b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd00b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 -- 172.21.15.50:0/2920673946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffeb00 msgr2=0x555bfd008e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffeb00 0x555bfd008e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.892+0000 7fa589e00640 1 -- 172.21.15.50:0/2920673946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfcc77210 con 0x555bfcffe1e0 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.893+0000 7fa589e00640 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfcffe5c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009f50 tx=0x7fa57c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eacc2e865ae03e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.893+0000 7fa576600640 1 -- 172.21.15.50:0/2920673946 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c004b20 con 0x555bfcffe1e0 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.893+0000 7fa576600640 1 -- 172.21.15.50:0/2920673946 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c004cc0 con 0x555bfcffe1e0 2024-09-12T03:30:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.893+0000 7fa576600640 1 -- 172.21.15.50:0/2920673946 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c005020 con 0x555bfcffe1e0 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa582000640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa56c0a2e20 0x7fa56c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2920673946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 msgr2=0x555bfcffe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfcffe5c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009f50 tx=0x7fa57c0118d0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa582000640 1 -- 172.21.15.50:0/464613389 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fa5780cfb70 con 0x7fa584103f50 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa589400640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa56c0a2e20 0x7fa56c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2920673946 shutdown_connections 2024-09-12T03:30:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd00b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffeb00 0x555bfd008e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2920673946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfcffe5c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2920673946 >> 172.21.15.50:0/2920673946 conn(0x555bfcffa750 msgr2=0x555bfcffab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2920673946 shutdown_connections 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.894+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2920673946 wait complete. 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.895+0000 7fa589400640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa56c0a2e20 0x7fa56c0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x555bfcffb5b0 tx=0x7fa57800c850 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.895+0000 7fa58ecc1740 1 -- start start 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.895+0000 7fa58ecc1740 1 Processor -- start 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd0062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd006820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfd0093a0 0x555bfd006d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555bfd00bf70 con 0x555bfcffe1e0 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555bfceed090 con 0x555bfcffeb00 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555bfd00c0f0 con 0x555bfd0093a0 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa58ecc1740 10 client.0 _ll_register_callbacks cb 0x555bfcf41f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa589e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd0062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa589e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd0062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60248/0 (socket says 172.21.15.50:60248) 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa589e00640 1 -- 172.21.15.50:0/145692394 learned_addr learned my addr 172.21.15.50:0/145692394 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa588a00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd006820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa589e00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfd0093a0 0x555bfd006d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa588a00640 1 -- 172.21.15.50:0/145692394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfd0093a0 msgr2=0x555bfd006d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa588a00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfd0093a0 0x555bfd006d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa588a00640 1 -- 172.21.15.50:0/145692394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 msgr2=0x555bfd0062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa588a00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd0062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.896+0000 7fa588a00640 1 -- 172.21.15.50:0/145692394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfcd43620 con 0x555bfcffeb00 2024-09-12T03:30:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa589e00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfd0093a0 0x555bfd006d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa588a00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd006820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa570009780 tx=0x7fa570004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ec23dcaa9ff44b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa56aa00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa56aa00640 1 -- 172.21.15.50:0/145692394 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570037040 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa56aa00640 1 -- 172.21.15.50:0/145692394 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fa570030550 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57c004cc0 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa56aa00640 1 -- 172.21.15.50:0/145692394 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa57004a050 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa58ecc1740 10 client.25505 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.897+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555bfd017b70 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa56aa00640 1 -- 172.21.15.50:0/145692394 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c004cc0 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa56aa00640 1 -- 172.21.15.50:0/145692394 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa570045030 con 0x555bfcffeb00 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa58ecc1740 10 client.25505 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa58ecc1740 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa58ecc1740 20 client.25505 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd019080 0x555bfd01b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa58ecc1740 4 client.25505 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bfcf71830 con 0x555bfd019080 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.898+0000 7fa589e00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd019080 0x555bfd01b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.899+0000 7fa589e00640 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd019080 0x555bfd01b4d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa467ea14b59447 server_cookie=20b50b369912c61a in_seq=0 out_seq=0 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.899+0000 7fa56aa00640 10 client.25505 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.899+0000 7fa56aa00640 1 -- 172.21.15.50:0/145692394 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bfcf71830 con 0x555bfd019080 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.899+0000 7fa56aa00640 10 client.25505 handle_command_reply: tid=0 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.899+0000 7fa58ecc1740 1 client.25505 shutdown 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 msgr2=0x555bfd006820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd006820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa570009780 tx=0x7fa570004270 comp rx=0 tx=0).stop 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 shutdown_connections 2024-09-12T03:30:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd019080 0x555bfd01b4d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfd0093a0 0x555bfd006d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd006820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa56aa00640 0 client.25505 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/145692394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd0062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 >> 172.21.15.50:0/145692394 conn(0x555bfcffa750 msgr2=0x555bfd0004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 shutdown_connections 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 1 -- 172.21.15.50:0/145692394 wait complete. 2024-09-12T03:30:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.900+0000 7fa58ecc1740 20 client.25505 trim_cache size 0 max 16384 2024-09-12T03:30:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.917+0000 7fa58ecc1740 1 Processor -- start 2024-09-12T03:30:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.917+0000 7fa58ecc1740 1 -- start start 2024-09-12T03:30:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd000c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 0x555bfd008140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd020da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b3b50 con 0x555bfcffe1e0 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b13c0 con 0x555bfd0093a0 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841af210 con 0x555bfcffeb00 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 0x555bfd008140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 0x555bfd008140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53286/0 (socket says 172.21.15.50:53286) 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 -- 172.21.15.50:0/1568904446 learned_addr learned my addr 172.21.15.50:0/1568904446 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa589e00640 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd000c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa589e00640 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd020da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 -- 172.21.15.50:0/1568904446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 msgr2=0x555bfd020da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd020da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 -- 172.21.15.50:0/1568904446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 msgr2=0x555bfd000c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd000c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.918+0000 7fa588a00640 1 -- 172.21.15.50:0/1568904446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfcffdf60 con 0x555bfcffeb00 2024-09-12T03:30:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.919+0000 7fa589e00640 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd020da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.919+0000 7fa588a00640 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 0x555bfd008140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa570004810 tx=0x7fa570000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad7fcfda8e62d25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.919+0000 7fa575c00640 1 -- 172.21.15.50:0/1568904446 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570053070 con 0x555bfcffeb00 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.919+0000 7fa575c00640 1 -- 172.21.15.50:0/1568904446 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa570004a00 con 0x555bfcffeb00 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.919+0000 7fa575c00640 1 -- 172.21.15.50:0/1568904446 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570037030 con 0x555bfcffeb00 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 -- 172.21.15.50:0/1568904446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 msgr2=0x555bfd008140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 0x555bfd008140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa570004810 tx=0x7fa570000bc0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 -- 172.21.15.50:0/1568904446 shutdown_connections 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfd0093a0 0x555bfd020da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffeb00 0x555bfd008140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/1568904446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfcffe1e0 0x555bfd000c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 -- 172.21.15.50:0/1568904446 >> 172.21.15.50:0/1568904446 conn(0x555bfcffa750 msgr2=0x555bfd007a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 -- 172.21.15.50:0/1568904446 shutdown_connections 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.920+0000 7fa58ecc1740 1 -- 172.21.15.50:0/1568904446 wait complete. 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.921+0000 7fa58ecc1740 1 -- start start 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.921+0000 7fa58ecc1740 1 Processor -- start 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.921+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfd011a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.921+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd002f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa58ecc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfd0093a0 0x555bfd003480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841af880 con 0x555bfd0093a0 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b0cf0 con 0x555bfcffeb00 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa58ecc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841aff50 con 0x555bfcffe1e0 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa58ecc1740 10 client.0 _ll_register_callbacks cb 0x7fa57c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd002f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd002f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:41114/0 (socket says 172.21.15.50:41114) 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 -- 172.21.15.50:0/2413369740 learned_addr learned my addr 172.21.15.50:0/2413369740 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa589e00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfd011a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa589e00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfd0093a0 0x555bfd003480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 -- 172.21.15.50:0/2413369740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 msgr2=0x555bfd011a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfd011a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 -- 172.21.15.50:0/2413369740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfd0093a0 msgr2=0x555bfd003480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfd0093a0 0x555bfd003480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa588a00640 1 -- 172.21.15.50:0/2413369740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfd003cb0 con 0x555bfcffeb00 2024-09-12T03:30:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.922+0000 7fa589e00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfd0093a0 0x555bfd003480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa588a00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd002f40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa570004950 tx=0x7fa570005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968f7bfbf1ee30d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa56aa00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa56aa00640 1 -- 172.21.15.50:0/2413369740 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570037050 con 0x555bfcffeb00 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa570004a00 con 0x555bfcffeb00 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa56aa00640 1 -- 172.21.15.50:0/2413369740 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fa570030550 con 0x555bfcffeb00 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa56aa00640 1 -- 172.21.15.50:0/2413369740 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa570053040 con 0x555bfcffeb00 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa58ecc1740 10 client.25511 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555bfd08c4d0 con 0x555bfcffeb00 2024-09-12T03:30:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.923+0000 7fa56aa00640 1 -- 172.21.15.50:0/2413369740 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa570004a00 con 0x555bfcffeb00 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa56aa00640 1 -- 172.21.15.50:0/2413369740 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa570056070 con 0x555bfcffeb00 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa58ecc1740 10 client.25511 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa58ecc1740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa58ecc1740 20 client.25511 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd08cc10 0x555bfd08f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa58ecc1740 4 client.25511 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bfcf6bb80 con 0x555bfd08cc10 2024-09-12T03:30:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.924+0000 7fa589e00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd08cc10 0x555bfd08f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.925+0000 7fa589e00640 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd08cc10 0x555bfd08f030 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cef4212cc227b4 server_cookie=ca63036fab646cd1 in_seq=0 out_seq=0 2024-09-12T03:30:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.925+0000 7fa56aa00640 10 client.25511 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa56aa00640 1 -- 172.21.15.50:0/2413369740 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bfcf6bb80 con 0x555bfd08cc10 2024-09-12T03:30:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa56aa00640 10 client.25511 handle_command_reply: tid=0 2024-09-12T03:30:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 client.25511 shutdown 2024-09-12T03:30:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 msgr2=0x555bfd002f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd002f40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa570004950 tx=0x7fa570005f80 comp rx=0 tx=0).stop 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 shutdown_connections 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bfd0093a0 0x555bfd003480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bfcffeb00 0x555bfd002f40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555bfd08cc10 0x555bfd08f030 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 --2- 172.21.15.50:0/2413369740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bfcffe1e0 0x555bfd011a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa56aa00640 0 client.25511 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.938+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 >> 172.21.15.50:0/2413369740 conn(0x555bfcffa750 msgr2=0x555bfcff9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.939+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 shutdown_connections 2024-09-12T03:30:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.939+0000 7fa58ecc1740 1 -- 172.21.15.50:0/2413369740 wait complete. 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.939+0000 7fa58ecc1740 20 client.25511 trim_cache size 0 max 16384 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa56c0a2e20 msgr2=0x7fa56c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa56c0a2e20 0x7fa56c0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x555bfcffb5b0 tx=0x7fa57800c850 comp rx=0 tx=0).stop 2024-09-12T03:30:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 msgr2=0x7fa58419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 0x7fa58419a5d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9c0 tx=0x7fa57800ee80 comp rx=0 tx=0).stop 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 shutdown_connections 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58419ee90 0x7fa58419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fa56c0a2e20 0x7fa56c0a52e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5841090a0 0x7fa58419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 --2- 172.21.15.50:0/464613389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584103f50 0x7fa58419a5d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 >> 172.21.15.50:0/464613389 conn(0x7fa5840ff9e0 msgr2=0x7fa584100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 shutdown_connections 2024-09-12T03:30:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:36.940+0000 7fa58be00640 1 -- 172.21.15.50:0/464613389 wait complete. 2024-09-12T03:30:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:36 smithi142 ceph-mon[35329]: pgmap v106: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-12T03:30:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:36 smithi082 ceph-mon[35046]: pgmap v106: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-12T03:30:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:36 smithi050 ceph-mon[30378]: pgmap v106: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-12T03:30:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:38 smithi142 ceph-mon[35329]: pgmap v107: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 387 op/s 2024-09-12T03:30:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:38 smithi082 ceph-mon[35046]: pgmap v107: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 387 op/s 2024-09-12T03:30:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:38 smithi050 ceph-mon[30378]: pgmap v107: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 387 op/s 2024-09-12T03:30:39.318 INFO:tasks.workunit.client.0.smithi050.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-09-12T03:30:39.318 INFO:tasks.workunit.client.0.smithi050.stdout:All 50000 operations completed A-OK! 2024-09-12T03:30:39.319 INFO:tasks.workunit.client.0.smithi050.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-09-12T03:30:39.321 INFO:tasks.workunit.client.0.smithi050.stdout:Seed set to 1 2024-09-12T03:30:39.466 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T03:30:39.467 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T03:30:39.468 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T03:30:39.468 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T03:30:39.468 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T03:30:39.470 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T03:30:39.473 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero length punch hole 2024-09-12T03:30:39.473 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T03:30:39.473 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero length punch hole 2024-09-12T03:30:39.474 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero length punch hole 2024-09-12T03:30:39.474 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero size read 2024-09-12T03:30:39.474 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero length punch hole 2024-09-12T03:30:39.474 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero size read 2024-09-12T03:30:39.474 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero length punch hole 2024-09-12T03:30:39.474 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x4310e9 2024-09-12T03:30:39.480 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x954a45 2024-09-12T03:30:39.543 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x9bc582 2024-09-12T03:30:39.586 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x9eecbc 2024-09-12T03:30:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:40 smithi142 ceph-mon[35329]: pgmap v108: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-12T03:30:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:40 smithi082 ceph-mon[35046]: pgmap v108: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-12T03:30:41.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:40 smithi050 ceph-mon[30378]: pgmap v108: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-12T03:30:42.894 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x9fdd84 2024-09-12T03:30:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:42 smithi142 ceph-mon[35329]: pgmap v109: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-12T03:30:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:42 smithi082 ceph-mon[35046]: pgmap v109: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-12T03:30:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:42 smithi050 ceph-mon[30378]: pgmap v109: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-12T03:30:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:44 smithi142 ceph-mon[35329]: pgmap v110: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 414 op/s 2024-09-12T03:30:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:44 smithi082 ceph-mon[35046]: pgmap v110: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 414 op/s 2024-09-12T03:30:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:44 smithi050 ceph-mon[30378]: pgmap v110: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 414 op/s 2024-09-12T03:30:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:46 smithi082 ceph-mon[35046]: pgmap v111: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-12T03:30:47.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:46 smithi050 ceph-mon[30378]: pgmap v111: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-12T03:30:47.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:46 smithi142 ceph-mon[35329]: pgmap v111: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-12T03:30:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:48 smithi082 ceph-mon[35046]: pgmap v112: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.7 MiB/s wr, 487 op/s 2024-09-12T03:30:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:48 smithi050 ceph-mon[30378]: pgmap v112: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.7 MiB/s wr, 487 op/s 2024-09-12T03:30:49.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:48 smithi142 ceph-mon[35329]: pgmap v112: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.7 MiB/s wr, 487 op/s 2024-09-12T03:30:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 -- 172.21.15.50:0/554403043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 msgr2=0x7f61e8104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 --2- 172.21.15.50:0/554403043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e8104330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009900 tx=0x7f61d802a410 comp rx=0 tx=0).stop 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 -- 172.21.15.50:0/554403043 shutdown_connections 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 --2- 172.21.15.50:0/554403043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61e81090a0 0x7f61e8110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 --2- 172.21.15.50:0/554403043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e8104900 0x7f61e8108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 --2- 172.21.15.50:0/554403043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e8104330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 -- 172.21.15.50:0/554403043 >> 172.21.15.50:0/554403043 conn(0x7f61e80ff9e0 msgr2=0x7f61e8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 -- 172.21.15.50:0/554403043 shutdown_connections 2024-09-12T03:30:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.069+0000 7f61ed400640 1 -- 172.21.15.50:0/554403043 wait complete. 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 Processor -- start 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 -- start start 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e81090a0 0x7f61e819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61e819ef00 0x7f61e819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8114130 con 0x7f61e819ef00 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8113fb0 con 0x7f61e8103f50 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.070+0000 7f61ed400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81142b0 con 0x7f61e81090a0 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:33892/0 (socket says 172.21.15.50:33892) 2024-09-12T03:30:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 -- 172.21.15.50:0/3363980707 learned_addr learned my addr 172.21.15.50:0/3363980707 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e5e00640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e81090a0 0x7f61e819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e7200640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61e819ef00 0x7f61e819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 -- 172.21.15.50:0/3363980707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e81090a0 msgr2=0x7f61e819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e81090a0 0x7f61e819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 -- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61e819ef00 msgr2=0x7f61e819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61e819ef00 0x7f61e819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 -- 172.21.15.50:0/3363980707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e819f9f0 con 0x7f61e8103f50 2024-09-12T03:30:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.071+0000 7f61e6800640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e819a640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61d8004770 tx=0x7f61d8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.072+0000 7f61df400640 1 -- 172.21.15.50:0/3363980707 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d803f070 con 0x7f61e8103f50 2024-09-12T03:30:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.072+0000 7f61df400640 1 -- 172.21.15.50:0/3363980707 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d8031ca0 con 0x7f61e8103f50 2024-09-12T03:30:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.072+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e819fc80 con 0x7f61e8103f50 2024-09-12T03:30:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.072+0000 7f61df400640 1 -- 172.21.15.50:0/3363980707 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8035030 con 0x7f61e8103f50 2024-09-12T03:30:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.072+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e81a74a0 con 0x7f61e8103f50 2024-09-12T03:30:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.075+0000 7f61df400640 1 -- 172.21.15.50:0/3363980707 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f61d8042070 con 0x7f61e8103f50 2024-09-12T03:30:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.075+0000 7f61f0205740 1 Processor -- start 2024-09-12T03:30:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.075+0000 7f61f0205740 1 -- start start 2024-09-12T03:30:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61f0205740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61f0205740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61f0205740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe34790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61f0205740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563e6fd8d100 con 0x563e6fe27b00 2024-09-12T03:30:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61f0205740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e6fd351b0 con 0x563e6fe271e0 2024-09-12T03:30:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61f0205740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563e6fd890c0 con 0x563e6fe323a0 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:33902/0 (socket says 172.21.15.50:33902) 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 -- 172.21.15.50:0/3939584896 learned_addr learned my addr 172.21.15.50:0/3939584896 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe34790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44158/0 (socket says 172.21.15.50:44158) 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 -- 172.21.15.50:0/3939584896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 msgr2=0x563e6fe34790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe34790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 -- 172.21.15.50:0/3939584896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 msgr2=0x563e6fe31e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.076+0000 7f61e7200640 1 -- 172.21.15.50:0/3939584896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6faa0210 con 0x563e6fe271e0 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.077+0000 7f61e7200640 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe275c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009f50 tx=0x7f61e00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452e9c3a596c2453 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.077+0000 7f61d3400640 1 -- 172.21.15.50:0/3939584896 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0004d20 con 0x563e6fe271e0 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.077+0000 7f61d3400640 1 -- 172.21.15.50:0/3939584896 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e0004050 con 0x563e6fe271e0 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.077+0000 7f61d3400640 1 -- 172.21.15.50:0/3939584896 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0005280 con 0x563e6fe271e0 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.077+0000 7f61df400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f61cc0a2e20 0x7f61cc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61df400640 1 -- 172.21.15.50:0/3363980707 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f61d80efd80 con 0x7f61e8103f50 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 -- 172.21.15.50:0/3939584896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 msgr2=0x563e6fe275c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe275c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009f50 tx=0x7f61e00118d0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61e6800640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f61cc0a2e20 0x7f61cc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 -- 172.21.15.50:0/3939584896 shutdown_connections 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe34790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 --2- 172.21.15.50:0/3939584896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe275c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 -- 172.21.15.50:0/3939584896 >> 172.21.15.50:0/3939584896 conn(0x563e6fe23750 msgr2=0x563e6fe23b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 -- 172.21.15.50:0/3939584896 shutdown_connections 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61f0205740 1 -- 172.21.15.50:0/3939584896 wait complete. 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.078+0000 7f61e6800640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f61cc0a2e20 0x7f61cc0a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x563e6fe245b0 tx=0x7f61d80f3830 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 -- start start 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 Processor -- start 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 0x563e6fe2f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe27b00 0x563e6fe2f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563e6fe34f70 con 0x563e6fe271e0 2024-09-12T03:30:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e6fd16090 con 0x563e6fe27b00 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563e6fe350f0 con 0x563e6fe323a0 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.079+0000 7f61f0205740 10 client.0 _ll_register_callbacks cb 0x563e6fd6af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 0x563e6fe2f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 0x563e6fe2f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44166/0 (socket says 172.21.15.50:44166) 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e7200640 1 -- 172.21.15.50:0/837412331 learned_addr learned my addr 172.21.15.50:0/837412331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe27b00 0x563e6fe2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e7200640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 -- 172.21.15.50:0/837412331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 msgr2=0x563e6fe2fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe2fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 -- 172.21.15.50:0/837412331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 msgr2=0x563e6fe2f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 0x563e6fe2f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 -- 172.21.15.50:0/837412331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6fb6c620 con 0x563e6fe27b00 2024-09-12T03:30:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e7200640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 0x563e6fe2f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61e5e00640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe27b00 0x563e6fe2f830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61e8100180 tx=0x7f61d4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6bdead7e218782 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.080+0000 7f61c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61c7e00640 1 -- 172.21.15.50:0/837412331 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4004020 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61c7e00640 1 -- 172.21.15.50:0/837412331 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f61d40041c0 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e0004050 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61c7e00640 1 -- 172.21.15.50:0/837412331 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61d40043c0 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 10 client.25523 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e6fe40bd0 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61c7e00640 1 -- 172.21.15.50:0/837412331 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e0004050 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61c7e00640 1 -- 172.21.15.50:0/837412331 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f61d4003bf0 con 0x563e6fe27b00 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 10 client.25523 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 20 client.25523 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6fe420e0 0x563e6fe44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 4 client.25523 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.081+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6fd9a830 con 0x563e6fe420e0 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.082+0000 7f61e7200640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6fe420e0 0x563e6fe44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.082+0000 7f61e7200640 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6fe420e0 0x563e6fe44510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd4a8afd1cd2fe7 server_cookie=14dfbe8d872fd477 in_seq=0 out_seq=0 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.082+0000 7f61c7e00640 10 client.25523 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.083+0000 7f61c7e00640 1 -- 172.21.15.50:0/837412331 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e6fd9a830 con 0x563e6fe420e0 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.083+0000 7f61c7e00640 10 client.25523 handle_command_reply: tid=0 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.083+0000 7f61f0205740 1 client.25523 shutdown 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe27b00 msgr2=0x563e6fe2f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe27b00 0x563e6fe2f830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61e8100180 tx=0x7f61d4012b60 comp rx=0 tx=0).stop 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 shutdown_connections 2024-09-12T03:30:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6fe420e0 0x563e6fe44510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6fe2fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe27b00 0x563e6fe2f830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 --2- 172.21.15.50:0/837412331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe271e0 0x563e6fe2f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61c7e00640 0 client.25523 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 >> 172.21.15.50:0/837412331 conn(0x563e6fe23750 msgr2=0x563e6fe294b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 shutdown_connections 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 1 -- 172.21.15.50:0/837412331 wait complete. 2024-09-12T03:30:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.084+0000 7f61f0205740 20 client.25523 trim_cache size 0 max 16384 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.101+0000 7f61f0205740 1 Processor -- start 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.101+0000 7f61f0205740 1 -- start start 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61f0205740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe271e0 0x563e6fe29c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61f0205740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61f0205740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe323a0 0x563e6fe49de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61f0205740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81b3ab0 con 0x563e6fe27b00 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61f0205740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81b1320 con 0x563e6fe323a0 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61f0205740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81af170 con 0x563e6fe271e0 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe271e0 0x563e6fe29c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe271e0 0x563e6fe29c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:47552/0 (socket says 172.21.15.50:47552) 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e7200640 1 -- 172.21.15.50:0/3525560164 learned_addr learned my addr 172.21.15.50:0/3525560164 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e7200640 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe323a0 0x563e6fe49de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e5e00640 1 -- 172.21.15.50:0/3525560164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe271e0 msgr2=0x563e6fe29c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e5e00640 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe271e0 0x563e6fe29c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e5e00640 1 -- 172.21.15.50:0/3525560164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe323a0 msgr2=0x563e6fe49de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e5e00640 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe323a0 0x563e6fe49de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.102+0000 7f61e5e00640 1 -- 172.21.15.50:0/3525560164 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6fe24800 con 0x563e6fe27b00 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.103+0000 7f61e5e00640 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f427f79607431ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.103+0000 7f61d2a00640 1 -- 172.21.15.50:0/3525560164 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d402e070 con 0x563e6fe27b00 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.103+0000 7f61d2a00640 1 -- 172.21.15.50:0/3525560164 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4005670 con 0x563e6fe27b00 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.103+0000 7f61d2a00640 1 -- 172.21.15.50:0/3525560164 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4003a40 con 0x563e6fe27b00 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 -- 172.21.15.50:0/3525560164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 msgr2=0x563e6fe31150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61d4000c00 tx=0x7f61d4009620 comp rx=0 tx=0).stop 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 -- 172.21.15.50:0/3525560164 shutdown_connections 2024-09-12T03:30:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe323a0 0x563e6fe49de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe31150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 --2- 172.21.15.50:0/3525560164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe271e0 0x563e6fe29c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 -- 172.21.15.50:0/3525560164 >> 172.21.15.50:0/3525560164 conn(0x563e6fe23750 msgr2=0x563e6fe30a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 -- 172.21.15.50:0/3525560164 shutdown_connections 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.104+0000 7f61f0205740 1 -- 172.21.15.50:0/3525560164 wait complete. 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.105+0000 7f61f0205740 1 -- start start 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.105+0000 7f61f0205740 1 Processor -- start 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.105+0000 7f61f0205740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.105+0000 7f61f0205740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61f0205740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6feabdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61f0205740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81af7e0 con 0x563e6fe27b00 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61f0205740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81b0c50 con 0x563e6fe271e0 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61f0205740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81afeb0 con 0x563e6fe323a0 2024-09-12T03:30:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61f0205740 10 client.0 _ll_register_callbacks cb 0x7f61d4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61e7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe36de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44176/0 (socket says 172.21.15.50:44176) 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61e5e00640 1 -- 172.21.15.50:0/2798416770 learned_addr learned my addr 172.21.15.50:0/2798416770 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61e7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe368a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:33928/0 (socket says 172.21.15.50:33928) 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.106+0000 7f61e7200640 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6feabdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61e5e00640 1 -- 172.21.15.50:0/2798416770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 msgr2=0x563e6feabdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61e5e00640 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6feabdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61e5e00640 1 -- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 msgr2=0x563e6fe368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61e5e00640 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61e5e00640 1 -- 172.21.15.50:0/2798416770 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6fe243e0 con 0x563e6fe27b00 2024-09-12T03:30:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61e5e00640 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe36de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005880 tx=0x7f61d4004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc37eb5c459dd09e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61c7e00640 1 -- 172.21.15.50:0/2798416770 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d402e040 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61c7e00640 1 -- 172.21.15.50:0/2798416770 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f61d40039e0 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.107+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4005670 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61c7e00640 1 -- 172.21.15.50:0/2798416770 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61d4003be0 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 10 client.15501 fetch_fsmap learned FSMap version 30 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e6fe1ed60 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61c7e00640 1 -- 172.21.15.50:0/2798416770 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4005670 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61c7e00640 1 -- 172.21.15.50:0/2798416770 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f61d401ca20 con 0x563e6fe27b00 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 20 client.15501 populate_metadata read hostname 'smithi050' 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6feada10 0x563e6feafe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 4 client.15501 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e6fd94b80 con 0x563e6feada10 2024-09-12T03:30:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.108+0000 7f61e7200640 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6feada10 0x563e6feafe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.109+0000 7f61e7200640 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6feada10 0x563e6feafe30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c365725165a612 server_cookie=cf7b15a7f5ae29d in_seq=0 out_seq=0 2024-09-12T03:30:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.109+0000 7f61c7e00640 10 client.15501 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.124+0000 7f61c7e00640 1 -- 172.21.15.50:0/2798416770 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e6fd94b80 con 0x563e6feada10 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.124+0000 7f61c7e00640 10 client.15501 handle_command_reply: tid=0 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.124+0000 7f61f0205740 1 client.15501 shutdown 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.124+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 msgr2=0x563e6fe36de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.124+0000 7f61f0205740 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe36de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005880 tx=0x7f61d4004210 comp rx=0 tx=0).stop 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 shutdown_connections 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563e6feada10 0x563e6feafe30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563e6fe323a0 0x563e6feabdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563e6fe27b00 0x563e6fe36de0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 --2- 172.21.15.50:0/2798416770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e6fe271e0 0x563e6fe368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 >> 172.21.15.50:0/2798416770 conn(0x563e6fe23750 msgr2=0x563e6fe2fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61c7e00640 0 client.15501 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 shutdown_connections 2024-09-12T03:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 1 -- 172.21.15.50:0/2798416770 wait complete. 2024-09-12T03:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.125+0000 7f61f0205740 20 client.15501 trim_cache size 0 max 16384 2024-09-12T03:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f61cc0a2e20 msgr2=0x7f61cc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f61cc0a2e20 0x7f61cc0a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x563e6fe245b0 tx=0x7f61d80f3830 comp rx=0 tx=0).stop 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 msgr2=0x7f61e819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e819a640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61d8004770 tx=0x7f61d8003c10 comp rx=0 tx=0).stop 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 shutdown_connections 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f61cc0a2e20 0x7f61cc0a52e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61e819ef00 0x7f61e819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e81090a0 0x7f61e819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 --2- 172.21.15.50:0/3363980707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e8103f50 0x7f61e819a640 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 >> 172.21.15.50:0/3363980707 conn(0x7f61e80ff9e0 msgr2=0x7f61e8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 shutdown_connections 2024-09-12T03:30:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:30:50.126+0000 7f61ed400640 1 -- 172.21.15.50:0/3363980707 wait complete. 2024-09-12T03:30:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:50 smithi082 ceph-mon[35046]: pgmap v113: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-12T03:30:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:50 smithi050 ceph-mon[30378]: pgmap v113: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-12T03:30:51.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:50 smithi142 ceph-mon[35329]: pgmap v113: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-12T03:30:53.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:52 smithi082 ceph-mon[35046]: pgmap v114: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-12T03:30:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:52 smithi050 ceph-mon[30378]: pgmap v114: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-12T03:30:53.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:52 smithi142 ceph-mon[35329]: pgmap v114: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-12T03:30:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:54 smithi082 ceph-mon[35046]: pgmap v115: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 570 op/s 2024-09-12T03:30:55.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:54 smithi050 ceph-mon[30378]: pgmap v115: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 570 op/s 2024-09-12T03:30:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:54 smithi142 ceph-mon[35329]: pgmap v115: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 570 op/s 2024-09-12T03:30:57.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:56 smithi082 ceph-mon[35046]: pgmap v116: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.2 MiB/s wr, 406 op/s 2024-09-12T03:30:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:56 smithi050 ceph-mon[30378]: pgmap v116: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.2 MiB/s wr, 406 op/s 2024-09-12T03:30:57.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:56 smithi142 ceph-mon[35329]: pgmap v116: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.2 MiB/s wr, 406 op/s 2024-09-12T03:30:59.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:30:58 smithi082 ceph-mon[35046]: pgmap v117: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 594 op/s 2024-09-12T03:30:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:30:58 smithi050 ceph-mon[30378]: pgmap v117: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 594 op/s 2024-09-12T03:30:59.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:30:58 smithi142 ceph-mon[35329]: pgmap v117: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 594 op/s 2024-09-12T03:31:01.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:00 smithi082 ceph-mon[35046]: pgmap v118: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-09-12T03:31:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:00 smithi050 ceph-mon[30378]: pgmap v118: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-09-12T03:31:01.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:00 smithi142 ceph-mon[35329]: pgmap v118: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-09-12T03:31:01.616 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x9ff773 2024-09-12T03:31:02.371 INFO:tasks.workunit.client.0.smithi050.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-09-12T03:31:03.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:02 smithi082 ceph-mon[35046]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-09-12T03:31:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:02 smithi050 ceph-mon[30378]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 -- 172.21.15.50:0/985946053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad0103400 msgr2=0x7f3ad010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 --2- 172.21.15.50:0/985946053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad0103400 0x7f3ad010b8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009900 tx=0x7f3abc02a480 comp rx=0 tx=0).stop 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 -- 172.21.15.50:0/985946053 shutdown_connections 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 --2- 172.21.15.50:0/985946053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad010bf00 0x7f3ad010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 --2- 172.21.15.50:0/985946053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad0103400 0x7f3ad010b8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 --2- 172.21.15.50:0/985946053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad0102ae0 0x7f3ad0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 -- 172.21.15.50:0/985946053 >> 172.21.15.50:0/985946053 conn(0x7f3ad00777c0 msgr2=0x7f3ad00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 -- 172.21.15.50:0/985946053 shutdown_connections 2024-09-12T03:31:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.261+0000 7f3acf200640 1 -- 172.21.15.50:0/985946053 wait complete. 2024-09-12T03:31:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 Processor -- start 2024-09-12T03:31:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 -- start start 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad0102ae0 0x7f3ad019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 0x7f3ad019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad010bf00 0x7f3ad019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0111d20 con 0x7f3ad0102ae0 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0111ba0 con 0x7f3ad0103400 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.262+0000 7f3acf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0111ea0 con 0x7f3ad010bf00 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 0x7f3ad019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 0x7f3ad019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42682/0 (socket says 172.21.15.50:42682) 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 -- 172.21.15.50:0/2622329045 learned_addr learned my addr 172.21.15.50:0/2622329045 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3ace800640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad010bf00 0x7f3ad019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acde00640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad0102ae0 0x7f3ad019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 -- 172.21.15.50:0/2622329045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad010bf00 msgr2=0x7f3ad019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad010bf00 0x7f3ad019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 -- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad0102ae0 msgr2=0x7f3ad019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad0102ae0 0x7f3ad019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 -- 172.21.15.50:0/2622329045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad019f480 con 0x7f3ad0103400 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.263+0000 7f3acd400640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 0x7f3ad019a9d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3abc02a9f0 tx=0x7f3abc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.264+0000 7f3ac6a00640 1 -- 172.21.15.50:0/2622329045 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc03f070 con 0x7f3ad0103400 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.264+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad019f710 con 0x7f3ad0103400 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.264+0000 7f3ac6a00640 1 -- 172.21.15.50:0/2622329045 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc031c90 con 0x7f3ad0103400 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.264+0000 7f3ac6a00640 1 -- 172.21.15.50:0/2622329045 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc035030 con 0x7f3ad0103400 2024-09-12T03:31:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.264+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a7000 con 0x7f3ad0103400 2024-09-12T03:31:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.268+0000 7f3ac6a00640 1 -- 172.21.15.50:0/2622329045 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3abc042070 con 0x7f3ad0103400 2024-09-12T03:31:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.268+0000 7f3ad78a5740 1 Processor -- start 2024-09-12T03:31:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.268+0000 7f3ad78a5740 1 -- start start 2024-09-12T03:31:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e03e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e049030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e04b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e41dfa4100 con 0x55e41e049570 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e41df4c1b0 con 0x55e41e03e3b0 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e41dfa00c0 con 0x55e41e03ecd0 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e03e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e03e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42694/0 (socket says 172.21.15.50:42694) 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 -- 172.21.15.50:0/2139407631 learned_addr learned my addr 172.21.15.50:0/2139407631 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3acde00640 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e049030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3acd400640 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e04b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 -- 172.21.15.50:0/2139407631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 msgr2=0x55e41e049030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e049030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 -- 172.21.15.50:0/2139407631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 msgr2=0x55e41e04b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e04b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.269+0000 7f3ace800640 1 -- 172.21.15.50:0/2139407631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41dcb7210 con 0x55e41e03e3b0 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3ace800640 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e03e790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009f50 tx=0x7f3ac80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99a909d3eb1319a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3ac6a00640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f3ab40a2dc0 0x7f3ab40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3abb000640 1 -- 172.21.15.50:0/2139407631 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8004d20 con 0x55e41e03e3b0 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3abb000640 1 -- 172.21.15.50:0/2139407631 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8004050 con 0x55e41e03e3b0 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3ac6a00640 1 -- 172.21.15.50:0/2622329045 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f3abc0f00a0 con 0x7f3ad0103400 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3abb000640 1 -- 172.21.15.50:0/2139407631 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8005280 con 0x55e41e03e3b0 2024-09-12T03:31:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.270+0000 7f3acde00640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f3ab40a2dc0 0x7f3ab40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 -- 172.21.15.50:0/2139407631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 msgr2=0x55e41e03e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e03e790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009f50 tx=0x7f3ac80118d0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3acde00640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f3ab40a2dc0 0x7f3ab40a5280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55e41e03b780 tx=0x7f3ac0005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 -- 172.21.15.50:0/2139407631 shutdown_connections 2024-09-12T03:31:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e04b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e049030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/2139407631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e03e790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 -- 172.21.15.50:0/2139407631 >> 172.21.15.50:0/2139407631 conn(0x55e41e03a920 msgr2=0x55e41e03ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 -- 172.21.15.50:0/2139407631 shutdown_connections 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.271+0000 7f3ad78a5740 1 -- 172.21.15.50:0/2139407631 wait complete. 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 -- start start 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 Processor -- start 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 0x55e41e046650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03ecd0 0x55e41e046b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e049570 0x55e41e0470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e41e04bf90 con 0x55e41e03ecd0 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e41df2d090 con 0x55e41e049570 2024-09-12T03:31:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e41e04c110 con 0x55e41e03e3b0 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3ad78a5740 10 client.0 _ll_register_callbacks cb 0x55e41df81f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3acde00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 0x55e41e046650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3acde00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 0x55e41e046650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:57818/0 (socket says 172.21.15.50:57818) 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.272+0000 7f3acde00640 1 -- 172.21.15.50:0/83817364 learned_addr learned my addr 172.21.15.50:0/83817364 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acd400640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03ecd0 0x55e41e046b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3ace800640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e049570 0x55e41e0470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acde00640 1 -- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e049570 msgr2=0x55e41e0470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acde00640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e049570 0x55e41e0470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acde00640 1 -- 172.21.15.50:0/83817364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03ecd0 msgr2=0x55e41e046b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acde00640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03ecd0 0x55e41e046b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acde00640 1 -- 172.21.15.50:0/83817364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41dd83620 con 0x55e41e03e3b0 2024-09-12T03:31:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3ace800640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e049570 0x55e41e0470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acd400640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03ecd0 0x55e41e046b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3acde00640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 0x55e41e046650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009730 tx=0x7f3ac0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf10f353076d33e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.273+0000 7f3aaf400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:31:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.274+0000 7f3aaf400640 1 -- 172.21.15.50:0/83817364 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac001f640 con 0x55e41e03e3b0 2024-09-12T03:31:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.274+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac8004050 con 0x55e41e03e3b0 2024-09-12T03:31:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.274+0000 7f3aaf400640 1 -- 172.21.15.50:0/83817364 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f3ac001f7e0 con 0x55e41e03e3b0 2024-09-12T03:31:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.274+0000 7f3aaf400640 1 -- 172.21.15.50:0/83817364 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ac001fa00 con 0x55e41e03e3b0 2024-09-12T03:31:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.274+0000 7f3ad78a5740 10 client.25531 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.274+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e41e057ea0 con 0x55e41e03e3b0 2024-09-12T03:31:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3aaf400640 1 -- 172.21.15.50:0/83817364 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8004050 con 0x55e41e03e3b0 2024-09-12T03:31:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3aaf400640 1 -- 172.21.15.50:0/83817364 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3ac0027070 con 0x55e41e03e3b0 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3ad78a5740 10 client.25531 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3ad78a5740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3ad78a5740 20 client.25531 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e059360 0x55e41e05b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3ad78a5740 4 client.25531 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.275+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e41dfb1830 con 0x55e41e059360 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.276+0000 7f3acd400640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e059360 0x55e41e05b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.276+0000 7f3acd400640 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e059360 0x55e41e05b750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c9148fc5a90f75 server_cookie=1af7dac8a09a766b in_seq=0 out_seq=0 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.276+0000 7f3aaf400640 10 client.25531 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.277+0000 7f3aaf400640 1 -- 172.21.15.50:0/83817364 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e41dfb1830 con 0x55e41e059360 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.277+0000 7f3aaf400640 10 client.25531 handle_command_reply: tid=0 2024-09-12T03:31:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.277+0000 7f3ad78a5740 1 client.25531 shutdown 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 msgr2=0x55e41e046650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 0x55e41e046650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009730 tx=0x7f3ac0004730 comp rx=0 tx=0).stop 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 shutdown_connections 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e049570 0x55e41e0470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03ecd0 0x55e41e046b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e059360 0x55e41e05b750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/83817364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03e3b0 0x55e41e046650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 >> 172.21.15.50:0/83817364 conn(0x55e41e03a920 msgr2=0x55e41e040630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3aaf400640 0 client.25531 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 shutdown_connections 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 1 -- 172.21.15.50:0/83817364 wait complete. 2024-09-12T03:31:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.278+0000 7f3ad78a5740 20 client.25531 trim_cache size 0 max 16384 2024-09-12T03:31:03.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:02 smithi142 ceph-mon[35329]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.3 MiB/s wr, 391 op/s 2024-09-12T03:31:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 Processor -- start 2024-09-12T03:31:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 -- start start 2024-09-12T03:31:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e040e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e044f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e045440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01b3980 con 0x55e41e049570 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01b11f0 con 0x55e41e03e3b0 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.298+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01af040 con 0x55e41e03ecd0 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e040e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e040e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42710/0 (socket says 172.21.15.50:42710) 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 -- 172.21.15.50:0/1235131794 learned_addr learned my addr 172.21.15.50:0/1235131794 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acd400640 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e044f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3ace800640 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e045440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 -- 172.21.15.50:0/1235131794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 msgr2=0x55e41e044f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e044f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 -- 172.21.15.50:0/1235131794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 msgr2=0x55e41e045440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e045440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 -- 172.21.15.50:0/1235131794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41e055280 con 0x55e41e03e3b0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3acde00640 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e040e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0032e30 tx=0x7f3ac0004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ddd8ac1804740d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.299+0000 7f3aba600640 1 -- 172.21.15.50:0/1235131794 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac001d4a0 con 0x55e41e03e3b0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3aba600640 1 -- 172.21.15.50:0/1235131794 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac001d640 con 0x55e41e03e3b0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3aba600640 1 -- 172.21.15.50:0/1235131794 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0032420 con 0x55e41e03e3b0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 -- 172.21.15.50:0/1235131794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 msgr2=0x55e41e040e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e040e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0032e30 tx=0x7f3ac0004690 comp rx=0 tx=0).stop 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 -- 172.21.15.50:0/1235131794 shutdown_connections 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e049570 0x55e41e045440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e03ecd0 0x55e41e044f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/1235131794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03e3b0 0x55e41e040e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 -- 172.21.15.50:0/1235131794 >> 172.21.15.50:0/1235131794 conn(0x55e41e03a920 msgr2=0x55e41e041930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 -- 172.21.15.50:0/1235131794 shutdown_connections 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.300+0000 7f3ad78a5740 1 -- 172.21.15.50:0/1235131794 wait complete. 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 -- start start 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 Processor -- start 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03e3b0 0x55e41e0517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 0x55e41e043920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e049570 0x55e41e043e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01af6b0 con 0x55e41e03e3b0 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01b0b20 con 0x55e41e03ecd0 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.301+0000 7f3ad78a5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01afd80 con 0x55e41e049570 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3ad78a5740 10 client.0 _ll_register_callbacks cb 0x7f3ac0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03e3b0 0x55e41e0517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 0x55e41e043920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 0x55e41e043920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42714/0 (socket says 172.21.15.50:42714) 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03e3b0 0x55e41e0517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60780/0 (socket says 172.21.15.50:60780) 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 -- 172.21.15.50:0/3441656479 learned_addr learned my addr 172.21.15.50:0/3441656479 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3ace800640 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e049570 0x55e41e043e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 -- 172.21.15.50:0/3441656479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e049570 msgr2=0x55e41e043e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e049570 0x55e41e043e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 -- 172.21.15.50:0/3441656479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03e3b0 msgr2=0x55e41e0517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03e3b0 0x55e41e0517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 -- 172.21.15.50:0/3441656479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41e0443a0 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3acd400640 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 0x55e41e043920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0bc270 tx=0x7f3abc03b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=202bc2273100b892 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3aaf400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.302+0000 7f3aaf400640 1 -- 172.21.15.50:0/3441656479 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc049040 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3aaf400640 1 -- 172.21.15.50:0/3441656479 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f3abc0ed6e0 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac001d640 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3aaf400640 1 -- 172.21.15.50:0/3441656479 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3abc031e40 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 10 client.25547 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e41e0c3030 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3aaf400640 1 -- 172.21.15.50:0/3441656479 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac001d640 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3aaf400640 1 -- 172.21.15.50:0/3441656479 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3abc0b8950 con 0x55e41e03ecd0 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 10 client.25547 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 20 client.25547 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e0c3580 0x55e41e0c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 4 client.25547 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e41dfabb80 con 0x55e41e0c3580 2024-09-12T03:31:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.303+0000 7f3acde00640 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e0c3580 0x55e41e0c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.304+0000 7f3acde00640 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e0c3580 0x55e41e0c59a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d723b413772d19 server_cookie=2d82eda92776dfd5 in_seq=0 out_seq=0 2024-09-12T03:31:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.304+0000 7f3aaf400640 10 client.25547 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.317+0000 7f3aaf400640 1 -- 172.21.15.50:0/3441656479 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e41dfabb80 con 0x55e41e0c3580 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.317+0000 7f3aaf400640 10 client.25547 handle_command_reply: tid=0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.317+0000 7f3ad78a5740 1 client.25547 shutdown 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 msgr2=0x55e41e043920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 0x55e41e043920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0bc270 tx=0x7f3abc03b6b0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 shutdown_connections 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55e41e0c3580 0x55e41e0c59a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e41e049570 0x55e41e043e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e41e03ecd0 0x55e41e043920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 --2- 172.21.15.50:0/3441656479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e41e03e3b0 0x55e41e0517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3aaf400640 0 client.25547 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 >> 172.21.15.50:0/3441656479 conn(0x55e41e03a920 msgr2=0x55e41e044ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 shutdown_connections 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 1 -- 172.21.15.50:0/3441656479 wait complete. 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.318+0000 7f3ad78a5740 20 client.25547 trim_cache size 0 max 16384 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f3ab40a2dc0 msgr2=0x7f3ab40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f3ab40a2dc0 0x7f3ab40a5280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55e41e03b780 tx=0x7f3ac0005f90 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 msgr2=0x7f3ad019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 0x7f3ad019a9d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3abc02a9f0 tx=0x7f3abc003b90 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 shutdown_connections 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ad010bf00 0x7f3ad019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0103400 0x7f3ad019a9d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f3ab40a2dc0 0x7f3ab40a5280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 --2- 172.21.15.50:0/2622329045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad0102ae0 0x7f3ad019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 >> 172.21.15.50:0/2622329045 conn(0x7f3ad00777c0 msgr2=0x7f3ad01006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 shutdown_connections 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:03.319+0000 7f3acf200640 1 -- 172.21.15.50:0/2622329045 wait complete. 2024-09-12T03:31:05.094 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:04 smithi142 ceph-mon[35329]: pgmap v120: 129 pgs: 129 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.4 MiB/s wr, 589 op/s 2024-09-12T03:31:05.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:04 smithi082 ceph-mon[35046]: pgmap v120: 129 pgs: 129 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.4 MiB/s wr, 589 op/s 2024-09-12T03:31:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:04 smithi050 ceph-mon[30378]: pgmap v120: 129 pgs: 129 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.4 MiB/s wr, 589 op/s 2024-09-12T03:31:07.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:06 smithi082 ceph-mon[35046]: pgmap v121: 129 pgs: 129 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:06 smithi050 ceph-mon[30378]: pgmap v121: 129 pgs: 129 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:07.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:06 smithi142 ceph-mon[35329]: pgmap v121: 129 pgs: 129 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:08 smithi082 ceph-mon[35046]: pgmap v122: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 574 op/s 2024-09-12T03:31:09.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:08 smithi050 ceph-mon[30378]: pgmap v122: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 574 op/s 2024-09-12T03:31:09.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:08 smithi142 ceph-mon[35329]: pgmap v122: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 574 op/s 2024-09-12T03:31:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:10 smithi082 ceph-mon[35046]: pgmap v123: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:11.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:10 smithi050 ceph-mon[30378]: pgmap v123: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:11.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:10 smithi142 ceph-mon[35329]: pgmap v123: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:12 smithi082 ceph-mon[35046]: pgmap v124: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:12 smithi050 ceph-mon[30378]: pgmap v124: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:13.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:12 smithi142 ceph-mon[35329]: pgmap v124: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-09-12T03:31:15.066 INFO:tasks.workunit.client.0.smithi050.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-09-12T03:31:15.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:14 smithi082 ceph-mon[35046]: pgmap v125: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.2 MiB/s wr, 562 op/s 2024-09-12T03:31:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:14 smithi050 ceph-mon[30378]: pgmap v125: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.2 MiB/s wr, 562 op/s 2024-09-12T03:31:15.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:14 smithi142 ceph-mon[35329]: pgmap v125: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.2 MiB/s wr, 562 op/s 2024-09-12T03:31:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 -- 172.21.15.50:0/611065999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c10bf00 msgr2=0x7f512c10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 --2- 172.21.15.50:0/611065999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c10bf00 0x7f512c10e370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5124009ca0 tx=0x7f512402a950 comp rx=0 tx=0).stop 2024-09-12T03:31:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 -- 172.21.15.50:0/611065999 shutdown_connections 2024-09-12T03:31:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 --2- 172.21.15.50:0/611065999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c10bf00 0x7f512c10e370 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 --2- 172.21.15.50:0/611065999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f512c103400 0x7f512c10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 --2- 172.21.15.50:0/611065999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f512c102ae0 0x7f512c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 -- 172.21.15.50:0/611065999 >> 172.21.15.50:0/611065999 conn(0x7f512c0777c0 msgr2=0x7f512c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 -- 172.21.15.50:0/611065999 shutdown_connections 2024-09-12T03:31:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.442+0000 7f5132600640 1 -- 172.21.15.50:0/611065999 wait complete. 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 Processor -- start 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 -- start start 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 0x7f512c100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f512c103400 0x7f512c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f512c10bf00 0x7f512c1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f512c111c90 con 0x7f512c103400 2024-09-12T03:31:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f512c111b10 con 0x7f512c102ae0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.443+0000 7f5132600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f512c111e10 con 0x7f512c10bf00 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 0x7f512c100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 0x7f512c100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57150/0 (socket says 172.21.15.50:57150) 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 -- 172.21.15.50:0/3699566771 learned_addr learned my addr 172.21.15.50:0/3699566771 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f512be00640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f512c103400 0x7f512c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131c00640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f512c10bf00 0x7f512c1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 -- 172.21.15.50:0/3699566771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f512c10bf00 msgr2=0x7f512c1011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f512c10bf00 0x7f512c1011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 -- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f512c103400 msgr2=0x7f512c100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f512c103400 0x7f512c100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 -- 172.21.15.50:0/3699566771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c1a66a0 con 0x7f512c102ae0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.444+0000 7f5131200640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 0x7f512c100730 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f512000ea60 tx=0x7f512000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.445+0000 7f5129600640 1 -- 172.21.15.50:0/3699566771 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000cde0 con 0x7f512c102ae0 2024-09-12T03:31:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.445+0000 7f5129600640 1 -- 172.21.15.50:0/3699566771 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51200035f0 con 0x7f512c102ae0 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.445+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1a6990 con 0x7f512c102ae0 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.445+0000 7f5129600640 1 -- 172.21.15.50:0/3699566771 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120010670 con 0x7f512c102ae0 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.445+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1a6ee0 con 0x7f512c102ae0 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 Processor -- start 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 -- start start 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b36790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b36cd0 0x563234b41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563234a9c100 con 0x563234b363b0 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563234a441b0 con 0x563234b41570 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5136a9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563234a980c0 con 0x563234b36cd0 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5129600640 1 -- 172.21.15.50:0/3699566771 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f51200037a0 con 0x7f512c102ae0 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5131c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b36790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f512be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f512be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b43960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57166/0 (socket says 172.21.15.50:57166) 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f512be00640 1 -- 172.21.15.50:0/4182057644 learned_addr learned my addr 172.21.15.50:0/4182057644 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.448+0000 7f5131200640 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b36cd0 0x563234b41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f512be00640 1 -- 172.21.15.50:0/4182057644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b36cd0 msgr2=0x563234b41030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f512be00640 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b36cd0 0x563234b41030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f512be00640 1 -- 172.21.15.50:0/4182057644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 msgr2=0x563234b36790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f512be00640 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b36790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f512be00640 1 -- 172.21.15.50:0/4182057644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632347af210 con 0x563234b41570 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f5131200640 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b36cd0 0x563234b41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f512be00640 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b43960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f511800bc10 tx=0x7f51180107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fa5765291617194 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f511e000640 1 -- 172.21.15.50:0/4182057644 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118005020 con 0x563234b41570 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f511e000640 1 -- 172.21.15.50:0/4182057644 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51180047a0 con 0x563234b41570 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.449+0000 7f511e000640 1 -- 172.21.15.50:0/4182057644 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180117a0 con 0x563234b41570 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5129600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f51140a2cf0 0x7f51140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5129600640 1 -- 172.21.15.50:0/3699566771 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f51200cfa90 con 0x7f512c102ae0 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 -- 172.21.15.50:0/4182057644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 msgr2=0x563234b43960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b43960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f511800bc10 tx=0x7f51180107a0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5131c00640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f51140a2cf0 0x7f51140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 -- 172.21.15.50:0/4182057644 shutdown_connections 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b43960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b36cd0 0x563234b41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4182057644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b36790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 -- 172.21.15.50:0/4182057644 >> 172.21.15.50:0/4182057644 conn(0x563234b32920 msgr2=0x563234b32d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.450+0000 7f5136a9e740 1 -- 172.21.15.50:0/4182057644 shutdown_connections 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.451+0000 7f5136a9e740 1 -- 172.21.15.50:0/4182057644 wait complete. 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.451+0000 7f5131c00640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f51140a2cf0 0x7f51140a51b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5124005e30 tx=0x7f5124005c50 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.451+0000 7f5136a9e740 1 -- start start 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.451+0000 7f5136a9e740 1 Processor -- start 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 0x563234b3ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b41570 0x563234b3f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563234b43ff0 con 0x563234b363b0 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563234a25090 con 0x563234b36cd0 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563234b44170 con 0x563234b41570 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5136a9e740 10 client.0 _ll_register_callbacks cb 0x563234a79f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 0x563234b3ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5131200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 0x563234b3ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57182/0 (socket says 172.21.15.50:57182) 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 -- 172.21.15.50:0/474943984 learned_addr learned my addr 172.21.15.50:0/474943984 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f5131c00640 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b41570 0x563234b3f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 -- 172.21.15.50:0/474943984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b41570 msgr2=0x563234b3f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b41570 0x563234b3f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 -- 172.21.15.50:0/474943984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 msgr2=0x563234b3e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b3e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.452+0000 7f512be00640 1 -- 172.21.15.50:0/474943984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323487b620 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f512be00640 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 0x563234b3ead0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5118005d00 tx=0x7f511800b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ef9405fc0397ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5112a00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5112a00640 1 -- 172.21.15.50:0/474943984 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118004020 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51180047a0 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5112a00640 1 -- 172.21.15.50:0/474943984 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f51180041c0 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5112a00640 1 -- 172.21.15.50:0/474943984 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51180043c0 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5136a9e740 10 client.25559 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.453+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563234b4fe60 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5112a00640 1 -- 172.21.15.50:0/474943984 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51180047a0 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5112a00640 1 -- 172.21.15.50:0/474943984 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5118003ac0 con 0x563234b36cd0 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5136a9e740 10 client.25559 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5136a9e740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5136a9e740 20 client.25559 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5136a9e740 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234b51320 0x563234b53710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5136a9e740 4 client.25559 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563234aa9830 con 0x563234b51320 2024-09-12T03:31:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.454+0000 7f5131200640 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234b51320 0x563234b53710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5131200640 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234b51320 0x563234b53710 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d65dcf03f955fe server_cookie=52932dfc2f149a24 in_seq=0 out_seq=0 2024-09-12T03:31:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5112a00640 10 client.25559 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5112a00640 1 -- 172.21.15.50:0/474943984 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563234aa9830 con 0x563234b51320 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5112a00640 10 client.25559 handle_command_reply: tid=0 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5136a9e740 1 client.25559 shutdown 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 msgr2=0x563234b3ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5136a9e740 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 0x563234b3ead0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5118005d00 tx=0x7f511800b920 comp rx=0 tx=0).stop 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 shutdown_connections 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.455+0000 7f5136a9e740 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234b51320 0x563234b53710 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b41570 0x563234b3f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b36cd0 0x563234b3ead0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 1 --2- 172.21.15.50:0/474943984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b363b0 0x563234b3e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5112a00640 0 client.25559 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 >> 172.21.15.50:0/474943984 conn(0x563234b32920 msgr2=0x563234b38680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 shutdown_connections 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 1 -- 172.21.15.50:0/474943984 wait complete. 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.456+0000 7f5136a9e740 20 client.25559 trim_cache size 0 max 16384 2024-09-12T03:31:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 Processor -- start 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 -- start start 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b38e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b58d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b3a40 con 0x563234b36cd0 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b12b0 con 0x563234b41570 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5136a9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1af100 con 0x563234b363b0 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5131200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b38e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5131200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b38e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32782/0 (socket says 172.21.15.50:32782) 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5131200640 1 -- 172.21.15.50:0/4195189035 learned_addr learned my addr 172.21.15.50:0/4195189035 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f512be00640 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b58d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.472+0000 7f5131c00640 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f5131200640 1 -- 172.21.15.50:0/4195189035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 msgr2=0x563234b592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f5131200640 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f5131200640 1 -- 172.21.15.50:0/4195189035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 msgr2=0x563234b58d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f5131200640 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b58d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f5131200640 1 -- 172.21.15.50:0/4195189035 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563234b2dc40 con 0x563234b363b0 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f512be00640 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b58d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f5131200640 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b38e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f512009bbb0 tx=0x7f512000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e5be1692af1b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f511d600640 1 -- 172.21.15.50:0/4195189035 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512009b580 con 0x563234b363b0 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f511d600640 1 -- 172.21.15.50:0/4195189035 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51200d7040 con 0x563234b363b0 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.473+0000 7f511d600640 1 -- 172.21.15.50:0/4195189035 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200cd0d0 con 0x563234b363b0 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 -- 172.21.15.50:0/4195189035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 msgr2=0x563234b38e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b38e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f512009bbb0 tx=0x7f512000ef90 comp rx=0 tx=0).stop 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 -- 172.21.15.50:0/4195189035 shutdown_connections 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b58d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 --2- 172.21.15.50:0/4195189035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b38e20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 -- 172.21.15.50:0/4195189035 >> 172.21.15.50:0/4195189035 conn(0x563234b32920 msgr2=0x563234b40500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 -- 172.21.15.50:0/4195189035 shutdown_connections 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 -- 172.21.15.50:0/4195189035 wait complete. 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 -- start start 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.474+0000 7f5136a9e740 1 Processor -- start 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b49ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b45000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b45580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1af770 con 0x563234b36cd0 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b0be0 con 0x563234b41570 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1afe40 con 0x563234b363b0 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5136a9e740 10 client.0 _ll_register_callbacks cb 0x7f511800ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5131c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b45580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5131200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b49ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f512be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b45000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5131200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b49ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:32794/0 (socket says 172.21.15.50:32794) 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5131200640 1 -- 172.21.15.50:0/3087704044 learned_addr learned my addr 172.21.15.50:0/3087704044 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5131200640 1 -- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 msgr2=0x563234b45580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.475+0000 7f5131200640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b45580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5131200640 1 -- 172.21.15.50:0/3087704044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 msgr2=0x563234b45000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5131200640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b45000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5131200640 1 -- 172.21.15.50:0/3087704044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563234b45db0 con 0x563234b363b0 2024-09-12T03:31:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5131c00640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b45580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f512be00640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b45000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5131200640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b49ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f512001f000 tx=0x7f512009b5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51d89ebd8201721 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5112a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5112a00640 1 -- 172.21.15.50:0/3087704044 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200a4890 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5112a00640 1 -- 172.21.15.50:0/3087704044 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f51200cd0d0 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51200d7040 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5112a00640 1 -- 172.21.15.50:0/3087704044 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f512009c320 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5136a9e740 10 client.25555 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563234bbbdf0 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.476+0000 7f5112a00640 1 -- 172.21.15.50:0/3087704044 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51200d7040 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5112a00640 1 -- 172.21.15.50:0/3087704044 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5120097970 con 0x563234b363b0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5136a9e740 10 client.25555 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5136a9e740 10 client.25555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5136a9e740 20 client.25555 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5136a9e740 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234bbc410 0x563234bbe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5136a9e740 4 client.25555 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563234aa3b80 con 0x563234bbc410 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f512be00640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234bbc410 0x563234bbe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f512be00640 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234bbc410 0x563234bbe830 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ef9ec45d8644cf server_cookie=c09f84b44f07ce1a in_seq=0 out_seq=0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.477+0000 7f5112a00640 10 client.25555 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5112a00640 1 -- 172.21.15.50:0/3087704044 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563234aa3b80 con 0x563234bbc410 2024-09-12T03:31:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5112a00640 10 client.25555 handle_command_reply: tid=0 2024-09-12T03:31:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 client.25555 shutdown 2024-09-12T03:31:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 msgr2=0x563234b49ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b49ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f512001f000 tx=0x7f512009b5f0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 shutdown_connections 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563234b41570 0x563234b45580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563234b36cd0 0x563234b45000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563234bbc410 0x563234bbe830 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 --2- 172.21.15.50:0/3087704044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563234b363b0 0x563234b49ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 >> 172.21.15.50:0/3087704044 conn(0x563234b32920 msgr2=0x563234b3fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.490+0000 7f5112a00640 0 client.25555 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.491+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 shutdown_connections 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.491+0000 7f5136a9e740 1 -- 172.21.15.50:0/3087704044 wait complete. 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.491+0000 7f5136a9e740 20 client.25555 trim_cache size 0 max 16384 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f51140a2cf0 msgr2=0x7f51140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f51140a2cf0 0x7f51140a51b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5124005e30 tx=0x7f5124005c50 comp rx=0 tx=0).stop 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 msgr2=0x7f512c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 0x7f512c100730 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f512000ea60 tx=0x7f512000ef20 comp rx=0 tx=0).stop 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 shutdown_connections 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f512c10bf00 0x7f512c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f51140a2cf0 0x7f51140a51b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f512c103400 0x7f512c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 --2- 172.21.15.50:0/3699566771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f512c102ae0 0x7f512c100730 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 >> 172.21.15.50:0/3699566771 conn(0x7f512c0777c0 msgr2=0x7f512c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 shutdown_connections 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:16.492+0000 7f5132600640 1 -- 172.21.15.50:0/3699566771 wait complete. 2024-09-12T03:31:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:16 smithi082 ceph-mon[35046]: pgmap v126: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 364 op/s 2024-09-12T03:31:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:16 smithi050 ceph-mon[30378]: pgmap v126: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 364 op/s 2024-09-12T03:31:17.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:16 smithi142 ceph-mon[35329]: pgmap v126: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 364 op/s 2024-09-12T03:31:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:18 smithi082 ceph-mon[35046]: pgmap v127: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.1 MiB/s wr, 538 op/s 2024-09-12T03:31:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:18 smithi050 ceph-mon[30378]: pgmap v127: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.1 MiB/s wr, 538 op/s 2024-09-12T03:31:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:18 smithi142 ceph-mon[35329]: pgmap v127: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.1 MiB/s wr, 538 op/s 2024-09-12T03:31:21.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:20 smithi082 ceph-mon[35046]: pgmap v128: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-09-12T03:31:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:20 smithi050 ceph-mon[30378]: pgmap v128: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-09-12T03:31:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:20 smithi142 ceph-mon[35329]: pgmap v128: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-09-12T03:31:22.103 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:21 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:21 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:21 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:22.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:21 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:31:22.122 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x9ffb01 2024-09-12T03:31:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:21 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:21 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:21 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:22.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:21 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:31:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:21 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:21 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:21 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:21 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:31:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:22 smithi082 ceph-mon[35046]: pgmap v129: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-09-12T03:31:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:22 smithi050 ceph-mon[30378]: pgmap v129: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-09-12T03:31:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:22 smithi142 ceph-mon[35329]: pgmap v129: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 350 op/s 2024-09-12T03:31:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:24 smithi082 ceph-mon[35046]: pgmap v130: 129 pgs: 129 active+clean; 64 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.0 MiB/s wr, 509 op/s 2024-09-12T03:31:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:24 smithi050 ceph-mon[30378]: pgmap v130: 129 pgs: 129 active+clean; 64 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.0 MiB/s wr, 509 op/s 2024-09-12T03:31:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:24 smithi142 ceph-mon[35329]: pgmap v130: 129 pgs: 129 active+clean; 64 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.0 MiB/s wr, 509 op/s 2024-09-12T03:31:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:26 smithi050 ceph-mon[30378]: pgmap v131: 129 pgs: 129 active+clean; 64 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.0 MiB/s wr, 333 op/s 2024-09-12T03:31:27.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:26 smithi142 ceph-mon[35329]: pgmap v131: 129 pgs: 129 active+clean; 64 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.0 MiB/s wr, 333 op/s 2024-09-12T03:31:27.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:26 smithi082 ceph-mon[35046]: pgmap v131: 129 pgs: 129 active+clean; 64 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.0 MiB/s wr, 333 op/s 2024-09-12T03:31:28.546 INFO:tasks.workunit.client.0.smithi050.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-09-12T03:31:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:28 smithi050 ceph-mon[30378]: pgmap v132: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.9 MiB/s wr, 506 op/s 2024-09-12T03:31:29.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:28 smithi142 ceph-mon[35329]: pgmap v132: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.9 MiB/s wr, 506 op/s 2024-09-12T03:31:29.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:28 smithi082 ceph-mon[35046]: pgmap v132: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.9 MiB/s wr, 506 op/s 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 -- 172.21.15.50:0/2877114201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc0104900 msgr2=0x7f9fc0108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 --2- 172.21.15.50:0/2877114201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc0104900 0x7f9fc0108940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009900 tx=0x7f9fac02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 -- 172.21.15.50:0/2877114201 shutdown_connections 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 --2- 172.21.15.50:0/2877114201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc01090a0 0x7f9fc0110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 --2- 172.21.15.50:0/2877114201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc0104900 0x7f9fc0108940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 --2- 172.21.15.50:0/2877114201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fc0103f50 0x7f9fc0104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 -- 172.21.15.50:0/2877114201 >> 172.21.15.50:0/2877114201 conn(0x7f9fc00ff9e0 msgr2=0x7f9fc0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.610+0000 7f9fc6600640 1 -- 172.21.15.50:0/2877114201 shutdown_connections 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.611+0000 7f9fc6600640 1 -- 172.21.15.50:0/2877114201 wait complete. 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.611+0000 7f9fc6600640 1 Processor -- start 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.611+0000 7f9fc6600640 1 -- start start 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fc6600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 0x7f9fc019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fc6600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fc01090a0 0x7f9fc019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fc6600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc019eea0 0x7f9fc019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fc6600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0114110 con 0x7f9fc01090a0 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fc6600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0113f90 con 0x7f9fc019eea0 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fc6600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0114290 con 0x7f9fc0103f50 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 0x7f9fc019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 0x7f9fc019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52714/0 (socket says 172.21.15.50:52714) 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fbf400640 1 -- 172.21.15.50:0/1227704860 learned_addr learned my addr 172.21.15.50:0/1227704860 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.612+0000 7f9fbea00640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fc01090a0 0x7f9fc019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fbf400640 1 -- 172.21.15.50:0/1227704860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc019eea0 msgr2=0x7f9fc019f350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:31:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fbf400640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc019eea0 0x7f9fc019f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fbf400640 1 -- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fc01090a0 msgr2=0x7f9fc019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fbf400640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fc01090a0 0x7f9fc019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fbf400640 1 -- 172.21.15.50:0/1227704860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc019f990 con 0x7f9fc0103f50 2024-09-12T03:31:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fbf400640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 0x7f9fc019a5e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000e9f0 tx=0x7f9fb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.613+0000 7f9fb7e00640 1 -- 172.21.15.50:0/1227704860 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000cd80 con 0x7f9fc0103f50 2024-09-12T03:31:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.614+0000 7f9fb7e00640 1 -- 172.21.15.50:0/1227704860 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb0010910 con 0x7f9fc0103f50 2024-09-12T03:31:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.614+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc019fc80 con 0x7f9fc0103f50 2024-09-12T03:31:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.614+0000 7f9fb7e00640 1 -- 172.21.15.50:0/1227704860 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0018980 con 0x7f9fc0103f50 2024-09-12T03:31:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.614+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a7490 con 0x7f9fc0103f50 2024-09-12T03:31:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.617+0000 7f9fc9483740 1 Processor -- start 2024-09-12T03:31:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.617+0000 7f9fc9483740 1 -- start start 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.617+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf3135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.617+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf313b00 0x5628cf31de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.617+0000 7f9fb7e00640 1 -- 172.21.15.50:0/1227704860 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9fb0010ac0 con 0x7f9fc0103f50 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf320790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fc9483740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5628cf279100 con 0x5628cf31e3a0 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fc9483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5628cf2211b0 con 0x5628cf3131e0 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fc9483740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5628cf2750c0 con 0x5628cf313b00 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf320790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf313b00 0x5628cf31de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf313b00 0x5628cf31de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52728/0 (socket says 172.21.15.50:52728) 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbf400640 1 -- 172.21.15.50:0/2365630345 learned_addr learned my addr 172.21.15.50:0/2365630345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf3135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf320790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57470/0 (socket says 172.21.15.50:57470) 2024-09-12T03:31:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbfe00640 1 -- 172.21.15.50:0/2365630345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf313b00 msgr2=0x5628cf31de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf313b00 0x5628cf31de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbfe00640 1 -- 172.21.15.50:0/2365630345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 msgr2=0x5628cf320790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf320790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.618+0000 7f9fbfe00640 1 -- 172.21.15.50:0/2365630345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628cef8c210 con 0x5628cf3131e0 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.619+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf3135c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009d00 tx=0x7f9fb800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a4a36389347766 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.619+0000 7f9fb4c00640 1 -- 172.21.15.50:0/2365630345 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8004b20 con 0x5628cf3131e0 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.619+0000 7f9fb7e00640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f9fa40a2e20 0x7f9fa40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.619+0000 7f9fb4c00640 1 -- 172.21.15.50:0/2365630345 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8003e50 con 0x5628cf3131e0 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.619+0000 7f9fb4c00640 1 -- 172.21.15.50:0/2365630345 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8004750 con 0x5628cf3131e0 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.619+0000 7f9fb7e00640 1 -- 172.21.15.50:0/1227704860 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f9fb00cfb10 con 0x7f9fc0103f50 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fbf400640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f9fa40a2e20 0x7f9fa40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 -- 172.21.15.50:0/2365630345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 msgr2=0x5628cf3135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf3135c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009d00 tx=0x7f9fb800e720 comp rx=0 tx=0).stop 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 -- 172.21.15.50:0/2365630345 shutdown_connections 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf320790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf313b00 0x5628cf31de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fbf400640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f9fa40a2e20 0x7f9fa40a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0098670 tx=0x7f9fb00d3660 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 --2- 172.21.15.50:0/2365630345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf3135c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 -- 172.21.15.50:0/2365630345 >> 172.21.15.50:0/2365630345 conn(0x5628cf30f750 msgr2=0x5628cf30fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 -- 172.21.15.50:0/2365630345 shutdown_connections 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.620+0000 7f9fc9483740 1 -- 172.21.15.50:0/2365630345 wait complete. 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.621+0000 7f9fc9483740 1 -- start start 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.621+0000 7f9fc9483740 1 Processor -- start 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf31b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf31b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf31e3a0 0x5628cf31bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5628cf320f70 con 0x5628cf313b00 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5628cf202090 con 0x5628cf31e3a0 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5628cf3210f0 con 0x5628cf3131e0 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fc9483740 10 client.0 _ll_register_callbacks cb 0x5628cf256f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf31b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf31e3a0 0x5628cf31bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf31b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57482/0 (socket says 172.21.15.50:57482) 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 -- 172.21.15.50:0/1013653923 learned_addr learned my addr 172.21.15.50:0/1013653923 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf31b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 -- 172.21.15.50:0/1013653923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 msgr2=0x5628cf31b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf31b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 -- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf31e3a0 msgr2=0x5628cf31bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf31e3a0 0x5628cf31bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.622+0000 7f9fbea00640 1 -- 172.21.15.50:0/1013653923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628cf058620 con 0x5628cf313b00 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fbea00640 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf31b7b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009900 tx=0x7f9fac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7505295b77a9873a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fa9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fa9600640 1 -- 172.21.15.50:0/1013653923 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035040 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8003e50 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fa9600640 1 -- 172.21.15.50:0/1013653923 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f9fac005450 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fa9600640 1 -- 172.21.15.50:0/1013653923 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fac04f050 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fc9483740 10 client.15549 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.623+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5628cf32cb40 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fa9600640 1 -- 172.21.15.50:0/1013653923 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8003e50 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fa9600640 1 -- 172.21.15.50:0/1013653923 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9fac003820 con 0x5628cf313b00 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fc9483740 10 client.15549 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fc9483740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fc9483740 20 client.15549 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fc9483740 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf32e020 0x5628cf330410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fc9483740 4 client.15549 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628cf286830 con 0x5628cf32e020 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.624+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf32e020 0x5628cf330410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.625+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf32e020 0x5628cf330410 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2944909608a0c server_cookie=462ed8874258fa9c in_seq=0 out_seq=0 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.625+0000 7f9fa9600640 10 client.15549 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fa9600640 1 -- 172.21.15.50:0/1013653923 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628cf286830 con 0x5628cf32e020 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fa9600640 10 client.15549 handle_command_reply: tid=0 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 client.15549 shutdown 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 msgr2=0x5628cf31b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf31b7b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009900 tx=0x7f9fac0043b0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 shutdown_connections 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf31e3a0 0x5628cf31bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf31b7b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf32e020 0x5628cf330410 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 --2- 172.21.15.50:0/1013653923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf31b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fa9600640 0 client.15549 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 >> 172.21.15.50:0/1013653923 conn(0x5628cf30f750 msgr2=0x5628cf315460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 shutdown_connections 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 1 -- 172.21.15.50:0/1013653923 wait complete. 2024-09-12T03:31:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.626+0000 7f9fc9483740 20 client.15549 trim_cache size 0 max 16384 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.643+0000 7f9fc9483740 1 Processor -- start 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 -- start start 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf30a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 0x5628cf31d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf335d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b3ac0 con 0x5628cf31e3a0 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b1330 con 0x5628cf313b00 2024-09-12T03:31:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fc9483740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01af180 con 0x5628cf3131e0 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf30a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 0x5628cf31d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf335d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 0x5628cf31d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:53852/0 (socket says 172.21.15.50:53852) 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fbea00640 1 -- 172.21.15.50:0/654226151 learned_addr learned my addr 172.21.15.50:0/654226151 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.644+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf30a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:52754/0 (socket says 172.21.15.50:52754) 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbea00640 1 -- 172.21.15.50:0/654226151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 msgr2=0x5628cf30a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbea00640 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf30a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbea00640 1 -- 172.21.15.50:0/654226151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 msgr2=0x5628cf335d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbea00640 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf335d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbea00640 1 -- 172.21.15.50:0/654226151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628cf310800 con 0x5628cf313b00 2024-09-12T03:31:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf30a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fbea00640 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 0x5628cf31d0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9fac003d90 tx=0x7f9fac005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1a35693ceaeb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fabe00640 1 -- 172.21.15.50:0/654226151 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac059070 con 0x5628cf313b00 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fabe00640 1 -- 172.21.15.50:0/654226151 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac041070 con 0x5628cf313b00 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.645+0000 7f9fabe00640 1 -- 172.21.15.50:0/654226151 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035030 con 0x5628cf313b00 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 -- 172.21.15.50:0/654226151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 msgr2=0x5628cf31d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 0x5628cf31d0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9fac003d90 tx=0x7f9fac005a40 comp rx=0 tx=0).stop 2024-09-12T03:31:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 -- 172.21.15.50:0/654226151 shutdown_connections 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf31e3a0 0x5628cf335d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf313b00 0x5628cf31d0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 --2- 172.21.15.50:0/654226151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf3131e0 0x5628cf30a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 -- 172.21.15.50:0/654226151 >> 172.21.15.50:0/654226151 conn(0x5628cf30f750 msgr2=0x5628cf31c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 -- 172.21.15.50:0/654226151 shutdown_connections 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 -- 172.21.15.50:0/654226151 wait complete. 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.646+0000 7f9fc9483740 1 -- start start 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 Processor -- start 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf321b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf322070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf31e3a0 0x5628cf326150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01af7f0 con 0x5628cf313b00 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b0c60 con 0x5628cf3131e0 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01afec0 con 0x5628cf31e3a0 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fc9483740 10 client.0 _ll_register_callbacks cb 0x7f9fac004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf321b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf321b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:53858/0 (socket says 172.21.15.50:53858) 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf322070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fbfe00640 1 -- 172.21.15.50:0/222395167 learned_addr learned my addr 172.21.15.50:0/222395167 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.647+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf31e3a0 0x5628cf326150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbea00640 1 -- 172.21.15.50:0/222395167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf31e3a0 msgr2=0x5628cf326150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbea00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf31e3a0 0x5628cf326150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbea00640 1 -- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 msgr2=0x5628cf321b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbea00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf321b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbea00640 1 -- 172.21.15.50:0/222395167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628cf313010 con 0x5628cf313b00 2024-09-12T03:31:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf321b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fbea00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf322070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fac02a910 tx=0x7f9fac005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a59d9fff8c3106 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fa9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fa9600640 1 -- 172.21.15.50:0/222395167 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac059040 con 0x5628cf313b00 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac041070 con 0x5628cf313b00 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.648+0000 7f9fa9600640 1 -- 172.21.15.50:0/222395167 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f9fac0035f0 con 0x5628cf313b00 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fa9600640 1 -- 172.21.15.50:0/222395167 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fac064050 con 0x5628cf313b00 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 10 client.15555 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5628cf398710 con 0x5628cf313b00 2024-09-12T03:31:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fa9600640 1 -- 172.21.15.50:0/222395167 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac041070 con 0x5628cf313b00 2024-09-12T03:31:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fa9600640 1 -- 172.21.15.50:0/222395167 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9fac031c80 con 0x5628cf313b00 2024-09-12T03:31:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 10 client.15555 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 20 client.15555 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf398d70 0x5628cf39b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 4 client.15555 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628cf280b80 con 0x5628cf398d70 2024-09-12T03:31:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.649+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf398d70 0x5628cf39b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.650+0000 7f9fbfe00640 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf398d70 0x5628cf39b190 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0766f53fd369a6b server_cookie=16287047192521e7 in_seq=0 out_seq=0 2024-09-12T03:31:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.650+0000 7f9fa9600640 10 client.15555 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.662+0000 7f9fa9600640 1 -- 172.21.15.50:0/222395167 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628cf280b80 con 0x5628cf398d70 2024-09-12T03:31:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.662+0000 7f9fa9600640 10 client.15555 handle_command_reply: tid=0 2024-09-12T03:31:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 client.15555 shutdown 2024-09-12T03:31:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 msgr2=0x5628cf322070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf322070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fac02a910 tx=0x7f9fac005b50 comp rx=0 tx=0).stop 2024-09-12T03:31:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 shutdown_connections 2024-09-12T03:31:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5628cf398d70 0x5628cf39b190 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628cf31e3a0 0x5628cf326150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fa9600640 0 client.15555 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628cf313b00 0x5628cf322070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 --2- 172.21.15.50:0/222395167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628cf3131e0 0x5628cf321b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 >> 172.21.15.50:0/222395167 conn(0x5628cf30f750 msgr2=0x5628cf30e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 shutdown_connections 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 1 -- 172.21.15.50:0/222395167 wait complete. 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.663+0000 7f9fc9483740 20 client.15555 trim_cache size 0 max 16384 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:31:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:31:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:31:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:31:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f9fa40a2e20 msgr2=0x7f9fa40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f9fa40a2e20 0x7f9fa40a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0098670 tx=0x7f9fb00d3660 comp rx=0 tx=0).stop 2024-09-12T03:31:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 msgr2=0x7f9fc019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 0x7f9fc019a5e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000e9f0 tx=0x7f9fb000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 shutdown_connections 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc019eea0 0x7f9fc019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f9fa40a2e20 0x7f9fa40a52e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fc01090a0 0x7f9fc019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 --2- 172.21.15.50:0/1227704860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc0103f50 0x7f9fc019a5e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.664+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 >> 172.21.15.50:0/1227704860 conn(0x7f9fc00ff9e0 msgr2=0x7f9fc0100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.665+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 shutdown_connections 2024-09-12T03:31:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:29.665+0000 7f9fc6600640 1 -- 172.21.15.50:0/1227704860 wait complete. 2024-09-12T03:31:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:30 smithi050 ceph-mon[30378]: pgmap v133: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-09-12T03:31:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:30 smithi142 ceph-mon[35329]: pgmap v133: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-09-12T03:31:31.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:30 smithi082 ceph-mon[35046]: pgmap v133: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-09-12T03:31:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:32 smithi050 ceph-mon[30378]: pgmap v134: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-09-12T03:31:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:32 smithi142 ceph-mon[35329]: pgmap v134: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-09-12T03:31:33.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:32 smithi082 ceph-mon[35046]: pgmap v134: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-09-12T03:31:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:34 smithi050 ceph-mon[30378]: pgmap v135: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 496 op/s 2024-09-12T03:31:35.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:34 smithi142 ceph-mon[35329]: pgmap v135: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 496 op/s 2024-09-12T03:31:35.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:34 smithi082 ceph-mon[35046]: pgmap v135: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 496 op/s 2024-09-12T03:31:37.156 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:36 smithi082 ceph-mon[35046]: pgmap v136: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-09-12T03:31:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:36 smithi050 ceph-mon[30378]: pgmap v136: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-09-12T03:31:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:36 smithi142 ceph-mon[35329]: pgmap v136: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-09-12T03:31:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:38 smithi050 ceph-mon[30378]: pgmap v137: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.8 MiB/s wr, 493 op/s 2024-09-12T03:31:39.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:38 smithi142 ceph-mon[35329]: pgmap v137: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.8 MiB/s wr, 493 op/s 2024-09-12T03:31:39.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:38 smithi082 ceph-mon[35046]: pgmap v137: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.8 MiB/s wr, 493 op/s 2024-09-12T03:31:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:40 smithi050 ceph-mon[30378]: pgmap v138: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-09-12T03:31:41.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:40 smithi142 ceph-mon[35329]: pgmap v138: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-09-12T03:31:41.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:40 smithi082 ceph-mon[35046]: pgmap v138: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-09-12T03:31:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 -- 172.21.15.50:0/438847877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 msgr2=0x7f3850074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 --2- 172.21.15.50:0/438847877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 0x7f3850074980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3844009900 tx=0x7f384402a480 comp rx=0 tx=0).stop 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 -- 172.21.15.50:0/438847877 shutdown_connections 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 --2- 172.21.15.50:0/438847877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f385010e3a0 0x7f3850110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 --2- 172.21.15.50:0/438847877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850074ec0 0x7f385010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 --2- 172.21.15.50:0/438847877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 0x7f3850074980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.785+0000 7f3857600640 1 -- 172.21.15.50:0/438847877 >> 172.21.15.50:0/438847877 conn(0x7f38500fd7e0 msgr2=0x7f38500ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.786+0000 7f3857600640 1 -- 172.21.15.50:0/438847877 shutdown_connections 2024-09-12T03:31:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.786+0000 7f3857600640 1 -- 172.21.15.50:0/438847877 wait complete. 2024-09-12T03:31:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.786+0000 7f3857600640 1 Processor -- start 2024-09-12T03:31:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 -- start start 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 0x7f385019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 0x7f385019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 0x7f385019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3850114110 con 0x7f38500745a0 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3850113f90 con 0x7f3850074ec0 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3857600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3850114290 con 0x7f385010e3a0 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f384fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 0x7f385019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3855600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 0x7f385019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3854c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 0x7f385019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f384fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 0x7f385019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59208/0 (socket says 172.21.15.50:59208) 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f3855600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 0x7f385019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50448/0 (socket says 172.21.15.50:50448) 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.787+0000 7f384fe00640 1 -- 172.21.15.50:0/3735732475 learned_addr learned my addr 172.21.15.50:0/3735732475 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384fe00640 1 -- 172.21.15.50:0/3735732475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 msgr2=0x7f385019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384fe00640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 0x7f385019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384fe00640 1 -- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 msgr2=0x7f385019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384fe00640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 0x7f385019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384fe00640 1 -- 172.21.15.50:0/3735732475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385019f5f0 con 0x7f3850074ec0 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f3855600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 0x7f385019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384fe00640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 0x7f385019ab40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e9f0 tx=0x7f383c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384d600640 1 -- 172.21.15.50:0/3735732475 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c00cd80 con 0x7f3850074ec0 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384d600640 1 -- 172.21.15.50:0/3735732475 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c005500 con 0x7f3850074ec0 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385019f8e0 con 0x7f3850074ec0 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f384d600640 1 -- 172.21.15.50:0/3735732475 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c010690 con 0x7f3850074ec0 2024-09-12T03:31:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.788+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501a7170 con 0x7f3850074ec0 2024-09-12T03:31:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 Processor -- start 2024-09-12T03:31:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 -- start start 2024-09-12T03:31:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f384d600640 1 -- 172.21.15.50:0/3735732475 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f383c0046c0 con 0x7f3850074ec0 2024-09-12T03:31:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e2335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e240790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f20e199100 con 0x55f20e233b00 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f20e1411b0 con 0x55f20e23e3a0 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.792+0000 7f385a46e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f20e1950c0 con 0x55f20e2331e0 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e2335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e2335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50464/0 (socket says 172.21.15.50:50464) 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 -- 172.21.15.50:0/2909750185 learned_addr learned my addr 172.21.15.50:0/2909750185 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3854c00640 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f384fe00640 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e240790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 -- 172.21.15.50:0/2909750185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 msgr2=0x55f20e240790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e240790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 -- 172.21.15.50:0/2909750185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 msgr2=0x55f20e23de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 -- 172.21.15.50:0/2909750185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20deac210 con 0x55f20e2331e0 2024-09-12T03:31:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.793+0000 7f3855600640 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e2335c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38480107a0 tx=0x7f3848010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b65a1a00d20486 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.794+0000 7f3841c00640 1 -- 172.21.15.50:0/2909750185 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848003b70 con 0x55f20e2331e0 2024-09-12T03:31:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.794+0000 7f3841c00640 1 -- 172.21.15.50:0/2909750185 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848010e50 con 0x55f20e2331e0 2024-09-12T03:31:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.794+0000 7f3841c00640 1 -- 172.21.15.50:0/2909750185 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848004730 con 0x55f20e2331e0 2024-09-12T03:31:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 -- 172.21.15.50:0/2909750185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 msgr2=0x55f20e2335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e2335c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38480107a0 tx=0x7f3848010c60 comp rx=0 tx=0).stop 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f384d600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f38380a2da0 0x7f38380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 -- 172.21.15.50:0/2909750185 shutdown_connections 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e240790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 --2- 172.21.15.50:0/2909750185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e2335c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 -- 172.21.15.50:0/2909750185 >> 172.21.15.50:0/2909750185 conn(0x55f20e22f750 msgr2=0x55f20e22fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f384d600640 1 -- 172.21.15.50:0/3735732475 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f383c0cfc00 con 0x7f3850074ec0 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 -- 172.21.15.50:0/2909750185 shutdown_connections 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f385a46e740 1 -- 172.21.15.50:0/2909750185 wait complete. 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.795+0000 7f3855600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f38380a2da0 0x7f38380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.796+0000 7f385a46e740 1 -- start start 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.796+0000 7f385a46e740 1 Processor -- start 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.796+0000 7f3855600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f38380a2da0 0x7f38380a5260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3848010530 tx=0x7f3848000f40 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e246a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 0x55f20e246f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e23e3a0 0x55f20e23ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f20e240f10 con 0x55f20e23e3a0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f20e122090 con 0x55f20e233b00 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f20e241090 con 0x55f20e2331e0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f385a46e740 10 client.0 _ll_register_callbacks cb 0x55f20e176f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 0x55f20e246f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f3854c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e246a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f3855600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e23e3a0 0x55f20e23ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 0x55f20e246f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59234/0 (socket says 172.21.15.50:59234) 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 -- 172.21.15.50:0/1144118937 learned_addr learned my addr 172.21.15.50:0/1144118937 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f3854c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e246a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50466/0 (socket says 172.21.15.50:50466) 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 -- 172.21.15.50:0/1144118937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 msgr2=0x55f20e246a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e246a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 -- 172.21.15.50:0/1144118937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e23e3a0 msgr2=0x55f20e23ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e23e3a0 0x55f20e23ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.797+0000 7f384fe00640 1 -- 172.21.15.50:0/1144118937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20df78620 con 0x55f20e233b00 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f384fe00640 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 0x55f20e246f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e9c0 tx=0x7f383c09bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65987995644c1486 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f3836000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:31:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f3836000640 1 -- 172.21.15.50:0/1144118937 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0cd440 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f3836000640 1 -- 172.21.15.50:0/1144118937 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f383c0a4570 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3848010e50 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f3836000640 1 -- 172.21.15.50:0/1144118937 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f383c09c490 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f385a46e740 10 client.25592 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f20e24ccc0 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.798+0000 7f3836000640 1 -- 172.21.15.50:0/1144118937 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848010e50 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f3836000640 1 -- 172.21.15.50:0/1144118937 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f383c09b1e0 con 0x55f20e233b00 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f385a46e740 10 client.25592 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f385a46e740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f385a46e740 20 client.25592 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f385a46e740 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e24e220 0x55f20e250610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f385a46e740 4 client.25592 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f20e1a6830 con 0x55f20e24e220 2024-09-12T03:31:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.799+0000 7f3854c00640 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e24e220 0x55f20e250610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.800+0000 7f3854c00640 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e24e220 0x55f20e250610 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6200998ea67f87 server_cookie=9791d688f0f2859e in_seq=0 out_seq=0 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.800+0000 7f3836000640 10 client.25592 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f3836000640 1 -- 172.21.15.50:0/1144118937 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f20e1a6830 con 0x55f20e24e220 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f3836000640 10 client.25592 handle_command_reply: tid=0 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 client.25592 shutdown 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 msgr2=0x55f20e246f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 0x55f20e246f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e9c0 tx=0x7f383c09bc20 comp rx=0 tx=0).stop 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 shutdown_connections 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e23e3a0 0x55f20e23ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e233b00 0x55f20e246f80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e24e220 0x55f20e250610 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 --2- 172.21.15.50:0/1144118937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e246a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 >> 172.21.15.50:0/1144118937 conn(0x55f20e22f750 msgr2=0x55f20e2354b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f3836000640 0 client.25592 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 shutdown_connections 2024-09-12T03:31:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 1 -- 172.21.15.50:0/1144118937 wait complete. 2024-09-12T03:31:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.801+0000 7f385a46e740 20 client.25592 trim_cache size 0 max 16384 2024-09-12T03:31:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 Processor -- start 2024-09-12T03:31:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 -- start start 2024-09-12T03:31:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e255f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e256440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38501af650 con 0x55f20e233b00 2024-09-12T03:31:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38501acec0 con 0x55f20e23e3a0 2024-09-12T03:31:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f385a46e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38501aad10 con 0x55f20e2331e0 2024-09-12T03:31:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f384fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f3855600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e256440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f3854c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e255f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f384fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45314/0 (socket says 172.21.15.50:45314) 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.818+0000 7f384fe00640 1 -- 172.21.15.50:0/2447775434 learned_addr learned my addr 172.21.15.50:0/2447775434 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f384fe00640 1 -- 172.21.15.50:0/2447775434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 msgr2=0x55f20e255f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f384fe00640 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e255f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f384fe00640 1 -- 172.21.15.50:0/2447775434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 msgr2=0x55f20e256440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f384fe00640 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e256440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f384fe00640 1 -- 172.21.15.50:0/2447775434 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20e242490 con 0x55f20e233b00 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f3854c00640 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e255f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f384fe00640 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23ce40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f383c0981f0 tx=0x7f383c0b4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7875ccbe19d42cba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.819+0000 7f3841200640 1 -- 172.21.15.50:0/2447775434 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c09bd90 con 0x55f20e233b00 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.820+0000 7f3841200640 1 -- 172.21.15.50:0/2447775434 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c0d31f0 con 0x55f20e233b00 2024-09-12T03:31:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.820+0000 7f3841200640 1 -- 172.21.15.50:0/2447775434 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0cd240 con 0x55f20e233b00 2024-09-12T03:31:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.820+0000 7f385a46e740 1 -- 172.21.15.50:0/2447775434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 msgr2=0x55f20e23ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.820+0000 7f385a46e740 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23ce40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f383c0981f0 tx=0x7f383c0b4f90 comp rx=0 tx=0).stop 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 -- 172.21.15.50:0/2447775434 shutdown_connections 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e256440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e23ce40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 --2- 172.21.15.50:0/2447775434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e255f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 -- 172.21.15.50:0/2447775434 >> 172.21.15.50:0/2447775434 conn(0x55f20e22f750 msgr2=0x55f20e2487c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 -- 172.21.15.50:0/2447775434 shutdown_connections 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.821+0000 7f385a46e740 1 -- 172.21.15.50:0/2447775434 wait complete. 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.822+0000 7f385a46e740 1 -- start start 2024-09-12T03:31:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.822+0000 7f385a46e740 1 Processor -- start 2024-09-12T03:31:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.822+0000 7f385a46e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e23cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.822+0000 7f385a46e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e238130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f385a46e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e238740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f385a46e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38501ab380 con 0x55f20e233b00 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f385a46e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38501ac7f0 con 0x55f20e23e3a0 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f385a46e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38501aba50 con 0x55f20e2331e0 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f385a46e740 10 client.0 _ll_register_callbacks cb 0x7f383c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e23cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f384fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e238130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3855600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e238740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e23cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:50480/0 (socket says 172.21.15.50:50480) 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 -- 172.21.15.50:0/337152246 learned_addr learned my addr 172.21.15.50:0/337152246 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 -- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 msgr2=0x55f20e238740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e238740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 -- 172.21.15.50:0/337152246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 msgr2=0x55f20e238130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e238130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.823+0000 7f3854c00640 1 -- 172.21.15.50:0/337152246 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20e238f40 con 0x55f20e2331e0 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f3854c00640 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e23cd30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3844000c00 tx=0x7f384402ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a45fa68079422f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f3836000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f3836000640 1 -- 172.21.15.50:0/337152246 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844035040 con 0x55f20e2331e0 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f3836000640 1 -- 172.21.15.50:0/337152246 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f3844005550 con 0x55f20e2331e0 2024-09-12T03:31:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c0d31f0 con 0x55f20e2331e0 2024-09-12T03:31:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f3836000640 1 -- 172.21.15.50:0/337152246 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3844066050 con 0x55f20e2331e0 2024-09-12T03:31:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f385a46e740 10 client.25591 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.824+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f20e2b9040 con 0x55f20e2331e0 2024-09-12T03:31:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f3836000640 1 -- 172.21.15.50:0/337152246 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c0d31f0 con 0x55f20e2331e0 2024-09-12T03:31:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f3836000640 1 -- 172.21.15.50:0/337152246 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3844031ad0 con 0x55f20e2331e0 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f385a46e740 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f385a46e740 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f385a46e740 20 client.25591 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f385a46e740 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e2b9820 0x55f20e2bbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f385a46e740 4 client.25591 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f20e1a0b80 con 0x55f20e2b9820 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.825+0000 7f384fe00640 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e2b9820 0x55f20e2bbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.826+0000 7f384fe00640 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e2b9820 0x55f20e2bbc40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d1d3845be6a40 server_cookie=f7bba32920bef519 in_seq=0 out_seq=0 2024-09-12T03:31:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.826+0000 7f3836000640 10 client.25591 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.841+0000 7f3836000640 1 -- 172.21.15.50:0/337152246 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f20e1a0b80 con 0x55f20e2b9820 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.841+0000 7f3836000640 10 client.25591 handle_command_reply: tid=0 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.841+0000 7f385a46e740 1 client.25591 shutdown 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 msgr2=0x55f20e23cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e23cd30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3844000c00 tx=0x7f384402ac00 comp rx=0 tx=0).stop 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 shutdown_connections 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f20e23e3a0 0x55f20e238740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f20e233b00 0x55f20e238130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f20e2b9820 0x55f20e2bbc40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 --2- 172.21.15.50:0/337152246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f20e2331e0 0x55f20e23cd30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f3836000640 0 client.25591 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 >> 172.21.15.50:0/337152246 conn(0x55f20e22f750 msgr2=0x55f20e247dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 shutdown_connections 2024-09-12T03:31:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 1 -- 172.21.15.50:0/337152246 wait complete. 2024-09-12T03:31:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.842+0000 7f385a46e740 20 client.25591 trim_cache size 0 max 16384 2024-09-12T03:31:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:31:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:31:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:31:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f38380a2da0 msgr2=0x7f38380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f38380a2da0 0x7f38380a5260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3848010530 tx=0x7f3848000f40 comp rx=0 tx=0).stop 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 msgr2=0x7f385019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 0x7f385019ab40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e9f0 tx=0x7f383c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 shutdown_connections 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f385010e3a0 0x7f385019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3850074ec0 0x7f385019ab40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f38380a2da0 0x7f38380a5260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 --2- 172.21.15.50:0/3735732475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38500745a0 0x7f385019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 >> 172.21.15.50:0/3735732475 conn(0x7f38500fd7e0 msgr2=0x7f38500fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 shutdown_connections 2024-09-12T03:31:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:42.843+0000 7f3857600640 1 -- 172.21.15.50:0/3735732475 wait complete. 2024-09-12T03:31:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:42 smithi050 ceph-mon[30378]: pgmap v139: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-09-12T03:31:43.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:42 smithi142 ceph-mon[35329]: pgmap v139: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-09-12T03:31:43.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:42 smithi082 ceph-mon[35046]: pgmap v139: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-09-12T03:31:43.382 INFO:tasks.workunit.client.0.smithi050.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-09-12T03:31:43.382 INFO:tasks.workunit.client.0.smithi050.stdout:All 50000 operations completed A-OK! 2024-09-12T03:31:43.386 INFO:tasks.workunit.client.0.smithi050.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-09-12T03:31:43.388 INFO:tasks.workunit.client.0.smithi050.stdout:Seed set to 1 2024-09-12T03:31:44.758 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T03:31:44.758 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T03:31:44.760 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T03:31:44.760 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T03:31:44.760 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T03:31:44.763 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T03:31:44.765 INFO:tasks.workunit.client.0.smithi050.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T03:31:44.765 INFO:tasks.workunit.client.0.smithi050.stdout:skipping zero size read 2024-09-12T03:31:44.782 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x1c37b90 2024-09-12T03:31:44.792 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x5bd330d 2024-09-12T03:31:44.816 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x637bf99 2024-09-12T03:31:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:44 smithi050 ceph-mon[30378]: pgmap v140: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 472 op/s 2024-09-12T03:31:45.227 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x63d9b9c 2024-09-12T03:31:45.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:44 smithi142 ceph-mon[35329]: pgmap v140: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 472 op/s 2024-09-12T03:31:45.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:44 smithi082 ceph-mon[35046]: pgmap v140: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 472 op/s 2024-09-12T03:31:47.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:46 smithi050 ceph-mon[30378]: pgmap v141: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2024-09-12T03:31:47.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:46 smithi142 ceph-mon[35329]: pgmap v141: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2024-09-12T03:31:47.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:46 smithi082 ceph-mon[35046]: pgmap v141: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2024-09-12T03:31:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:48 smithi050 ceph-mon[30378]: pgmap v142: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 424 op/s 2024-09-12T03:31:49.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:48 smithi142 ceph-mon[35329]: pgmap v142: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 424 op/s 2024-09-12T03:31:49.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:48 smithi082 ceph-mon[35046]: pgmap v142: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 424 op/s 2024-09-12T03:31:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:50 smithi050 ceph-mon[30378]: pgmap v143: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T03:31:51.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:50 smithi142 ceph-mon[35329]: pgmap v143: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T03:31:51.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:50 smithi082 ceph-mon[35046]: pgmap v143: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T03:31:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:52 smithi050 ceph-mon[30378]: pgmap v144: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T03:31:53.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:52 smithi142 ceph-mon[35329]: pgmap v144: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T03:31:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:52 smithi082 ceph-mon[35046]: pgmap v144: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T03:31:55.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:54 smithi050 ceph-mon[30378]: pgmap v145: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.9 MiB/s wr, 458 op/s 2024-09-12T03:31:55.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:54 smithi142 ceph-mon[35329]: pgmap v145: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.9 MiB/s wr, 458 op/s 2024-09-12T03:31:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:54 smithi082 ceph-mon[35046]: pgmap v145: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.9 MiB/s wr, 458 op/s 2024-09-12T03:31:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 -- 172.21.15.50:0/541517671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac103f50 msgr2=0x7f7eac104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 --2- 172.21.15.50:0/541517671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac103f50 0x7f7eac104330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009900 tx=0x7f7ea002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 -- 172.21.15.50:0/541517671 shutdown_connections 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 --2- 172.21.15.50:0/541517671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eac1090a0 0x7f7eac110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 --2- 172.21.15.50:0/541517671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eac104900 0x7f7eac108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 --2- 172.21.15.50:0/541517671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac103f50 0x7f7eac104330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.982+0000 7f7eb4800640 1 -- 172.21.15.50:0/541517671 >> 172.21.15.50:0/541517671 conn(0x7f7eac0ff9e0 msgr2=0x7f7eac101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.983+0000 7f7eb4800640 1 -- 172.21.15.50:0/541517671 shutdown_connections 2024-09-12T03:31:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.983+0000 7f7eb4800640 1 -- 172.21.15.50:0/541517671 wait complete. 2024-09-12T03:31:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 Processor -- start 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 -- start start 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eac103f50 0x7f7eac19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 0x7f7eac19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eac19ef00 0x7f7eac19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac114130 con 0x7f7eac103f50 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac113fb0 con 0x7f7eac1090a0 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb4800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1142b0 con 0x7f7eac19ef00 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 0x7f7eac19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eac103f50 0x7f7eac19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.984+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 0x7f7eac19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:58976/0 (socket says 172.21.15.50:58976) 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 -- 172.21.15.50:0/3204544657 learned_addr learned my addr 172.21.15.50:0/3204544657 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb2800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eac19ef00 0x7f7eac19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 -- 172.21.15.50:0/3204544657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eac19ef00 msgr2=0x7f7eac19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eac19ef00 0x7f7eac19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 -- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eac103f50 msgr2=0x7f7eac19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eac103f50 0x7f7eac19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 -- 172.21.15.50:0/3204544657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac19f9f0 con 0x7f7eac1090a0 2024-09-12T03:31:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.985+0000 7f7eb1400640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 0x7f7eac19ab80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e9f0 tx=0x7f7e9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.986+0000 7f7ea6a00640 1 -- 172.21.15.50:0/3204544657 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00ccc0 con 0x7f7eac1090a0 2024-09-12T03:31:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.986+0000 7f7ea6a00640 1 -- 172.21.15.50:0/3204544657 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c005500 con 0x7f7eac1090a0 2024-09-12T03:31:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.986+0000 7f7ea6a00640 1 -- 172.21.15.50:0/3204544657 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c010690 con 0x7f7eac1090a0 2024-09-12T03:31:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.986+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac19fce0 con 0x7f7eac1090a0 2024-09-12T03:31:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.986+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac1a7480 con 0x7f7eac1090a0 2024-09-12T03:31:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.995+0000 7f7ea6a00640 1 -- 172.21.15.50:0/3204544657 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7e9c0046c0 con 0x7f7eac1090a0 2024-09-12T03:31:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.995+0000 7f7eb764a740 1 Processor -- start 2024-09-12T03:31:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.995+0000 7f7eb764a740 1 -- start start 2024-09-12T03:31:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 0x558cc12965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc12a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb764a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558cc11fc100 con 0x558cc12a13a0 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb764a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558cc11a41b0 con 0x558cc1296b00 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb764a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558cc11f80c0 con 0x558cc12961e0 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 0x558cc12965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 0x558cc12965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:35716/0 (socket says 172.21.15.50:35716) 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb2800640 1 -- 172.21.15.50:0/2484512875 learned_addr learned my addr 172.21.15.50:0/2484512875 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.996+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc12a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb2800640 1 -- 172.21.15.50:0/2484512875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 msgr2=0x558cc12a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb2800640 1 -- 172.21.15.50:0/2484512875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 msgr2=0x558cc12a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc12a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb2800640 1 -- 172.21.15.50:0/2484512875 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc0f0f210 con 0x558cc12961e0 2024-09-12T03:31:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc12a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 0x558cc12965c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009f50 tx=0x7f7ea80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=209ab8af993bd5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7ea6a00640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7e940a2e20 0x7f7e940a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7e9b000640 1 -- 172.21.15.50:0/2484512875 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8004d20 con 0x558cc12961e0 2024-09-12T03:31:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7ea6a00640 1 -- 172.21.15.50:0/3204544657 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f7e9c0cfac0 con 0x7f7eac1090a0 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7e9b000640 1 -- 172.21.15.50:0/2484512875 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea8004050 con 0x558cc12961e0 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.997+0000 7f7e9b000640 1 -- 172.21.15.50:0/2484512875 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8005280 con 0x558cc12961e0 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.999+0000 7f7eb764a740 1 -- 172.21.15.50:0/2484512875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 msgr2=0x558cc12965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.999+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 0x558cc12965c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009f50 tx=0x7f7ea80118d0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.999+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7e940a2e20 0x7f7e940a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.999+0000 7f7eb764a740 1 -- 172.21.15.50:0/2484512875 shutdown_connections 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.999+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc12a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:55.999+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.000+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2484512875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12961e0 0x558cc12965c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.000+0000 7f7eb764a740 1 -- 172.21.15.50:0/2484512875 >> 172.21.15.50:0/2484512875 conn(0x558cc1292750 msgr2=0x558cc1292b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.000+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7e940a2e20 0x7f7e940a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c00 tx=0x7f7ea0005b50 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.000+0000 7f7eb764a740 1 -- 172.21.15.50:0/2484512875 shutdown_connections 2024-09-12T03:31:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.000+0000 7f7eb764a740 1 -- 172.21.15.50:0/2484512875 wait complete. 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.001+0000 7f7eb764a740 1 -- start start 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.001+0000 7f7eb764a740 1 Processor -- start 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc129e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc1296b00 0x558cc129e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc129edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558cc12a3f70 con 0x558cc12a13a0 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558cc1185090 con 0x558cc12961e0 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558cc12a40f0 con 0x558cc1296b00 2024-09-12T03:31:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb764a740 10 client.0 _ll_register_callbacks cb 0x558cc11d9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.002+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc129e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc129e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:58988/0 (socket says 172.21.15.50:58988) 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 -- 172.21.15.50:0/2459442902 learned_addr learned my addr 172.21.15.50:0/2459442902 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc129edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc1296b00 0x558cc129e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 -- 172.21.15.50:0/2459442902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc1296b00 msgr2=0x558cc129e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc1296b00 0x558cc129e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 -- 172.21.15.50:0/2459442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 msgr2=0x558cc129edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc129edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1e00640 1 -- 172.21.15.50:0/2459442902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc0fdb620 con 0x558cc12961e0 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.003+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc1296b00 0x558cc129e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc129edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc129e340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0033000 tx=0x7f7ea0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e45049e5151347d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7e8f400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7e8f400640 1 -- 172.21.15.50:0/2459442902 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea004e070 con 0x558cc12961e0 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7e8f400640 1 -- 172.21.15.50:0/2459442902 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f7ea0004b90 con 0x558cc12961e0 2024-09-12T03:31:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea8004050 con 0x558cc12961e0 2024-09-12T03:31:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.004+0000 7f7e8f400640 1 -- 172.21.15.50:0/2459442902 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ea005a050 con 0x558cc12961e0 2024-09-12T03:31:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.005+0000 7f7eb764a740 10 client.25610 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.005+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558cc12afb40 con 0x558cc12961e0 2024-09-12T03:31:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.005+0000 7f7e8f400640 1 -- 172.21.15.50:0/2459442902 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea8004050 con 0x558cc12961e0 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.005+0000 7f7e8f400640 1 -- 172.21.15.50:0/2459442902 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ea0003780 con 0x558cc12961e0 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb764a740 10 client.25610 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb764a740 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb764a740 20 client.25610 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc12b1000 0x558cc12b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb764a740 4 client.25610 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc1209830 con 0x558cc12b1000 2024-09-12T03:31:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.006+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc12b1000 0x558cc12b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.007+0000 7f7e8f400640 10 client.25610 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.007+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc12b1000 0x558cc12b33f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0965168171f1619 server_cookie=e11e47931164a28b in_seq=0 out_seq=0 2024-09-12T03:31:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7e8f400640 1 -- 172.21.15.50:0/2459442902 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cc1209830 con 0x558cc12b1000 2024-09-12T03:31:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7e8f400640 10 client.25610 handle_command_reply: tid=0 2024-09-12T03:31:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 client.25610 shutdown 2024-09-12T03:31:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 msgr2=0x558cc129e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc129e340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0033000 tx=0x7f7ea0009260 comp rx=0 tx=0).stop 2024-09-12T03:31:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 shutdown_connections 2024-09-12T03:31:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12a13a0 0x558cc129edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc12b1000 0x558cc12b33f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc1296b00 0x558cc129e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2459442902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc129e340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 >> 172.21.15.50:0/2459442902 conn(0x558cc1292750 msgr2=0x558cc1298460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.008+0000 7f7e8f400640 0 client.25610 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.009+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 shutdown_connections 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.009+0000 7f7eb764a740 1 -- 172.21.15.50:0/2459442902 wait complete. 2024-09-12T03:31:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.009+0000 7f7eb764a740 20 client.25610 trim_cache size 0 max 16384 2024-09-12T03:31:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 Processor -- start 2024-09-12T03:31:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 -- start start 2024-09-12T03:31:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc1298c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc1296b00 0x558cc12b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1af640 con 0x558cc1296b00 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1aceb0 con 0x558cc12961e0 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb764a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1aad20 con 0x558cc12a13a0 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc1298c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc1296b00 0x558cc12b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc1298c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:58994/0 (socket says 172.21.15.50:58994) 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb1e00640 1 -- 172.21.15.50:0/3019306108 learned_addr learned my addr 172.21.15.50:0/3019306108 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.035+0000 7f7eb2800640 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1400640 1 -- 172.21.15.50:0/3019306108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 msgr2=0x558cc12b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1400640 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1400640 1 -- 172.21.15.50:0/3019306108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 msgr2=0x558cc1298c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1400640 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc1298c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1400640 1 -- 172.21.15.50:0/3019306108 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc128dd60 con 0x558cc1296b00 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc1298c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7eb1400640 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc1296b00 0x558cc12b8d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0cce80 tx=0x7f7e9c0d1fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57fa1294660a074 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7e9a600640 1 -- 172.21.15.50:0/3019306108 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0b1070 con 0x558cc1296b00 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7e9a600640 1 -- 172.21.15.50:0/3019306108 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c0b4e50 con 0x558cc1296b00 2024-09-12T03:31:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.036+0000 7f7e9a600640 1 -- 172.21.15.50:0/3019306108 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0a4430 con 0x558cc1296b00 2024-09-12T03:31:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 -- 172.21.15.50:0/3019306108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc1296b00 msgr2=0x558cc12b8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc1296b00 0x558cc12b8d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0cce80 tx=0x7f7e9c0d1fd0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 -- 172.21.15.50:0/3019306108 shutdown_connections 2024-09-12T03:31:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc1296b00 0x558cc12b8d00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 --2- 172.21.15.50:0/3019306108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc12961e0 0x558cc1298c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 -- 172.21.15.50:0/3019306108 >> 172.21.15.50:0/3019306108 conn(0x558cc1292750 msgr2=0x558cc1299680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 -- 172.21.15.50:0/3019306108 shutdown_connections 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.037+0000 7f7eb764a740 1 -- 172.21.15.50:0/3019306108 wait complete. 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 -- start start 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 Processor -- start 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12961e0 0x558cc129c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1ab370 con 0x558cc12961e0 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1ac7e0 con 0x558cc1296b00 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1aba40 con 0x558cc12a13a0 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.038+0000 7f7eb764a740 10 client.0 _ll_register_callbacks cb 0x7f7ea0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12961e0 0x558cc129c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59000/0 (socket says 172.21.15.50:59000) 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb1400640 1 -- 172.21.15.50:0/2066517681 learned_addr learned my addr 172.21.15.50:0/2066517681 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 -- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 msgr2=0x558cc12a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 -- 172.21.15.50:0/2066517681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12961e0 msgr2=0x558cc129c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12961e0 0x558cc129c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 -- 172.21.15.50:0/2066517681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc129a690 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb1400640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12961e0 0x558cc129c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7eb2800640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12a9ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8004740 tx=0x7f7ea80048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b2e6f91febf802 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.039+0000 7f7e8f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7e8f400640 1 -- 172.21.15.50:0/2066517681 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8003cc0 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7e8f400640 1 -- 172.21.15.50:0/2066517681 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f7ea8004050 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c0b4e50 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7e8f400640 1 -- 172.21.15.50:0/2066517681 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ea803f050 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 10 client.25609 fetch_fsmap learned FSMap version 30 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558cc131b090 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7e8f400640 1 -- 172.21.15.50:0/2066517681 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c0b4e50 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7e8f400640 1 -- 172.21.15.50:0/2066517681 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ea800fbc0 con 0x558cc12a13a0 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 10 client.25609 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:31:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 10 client.25609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 20 client.25609 populate_metadata read hostname 'smithi050' 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc131b620 0x558cc131da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 4 client.25609 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.040+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cc1203b80 con 0x558cc131b620 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.041+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc131b620 0x558cc131da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.041+0000 7f7eb1e00640 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc131b620 0x558cc131da40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9cab5bdcfb0023 server_cookie=32e2653f8f417a59 in_seq=0 out_seq=0 2024-09-12T03:31:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.041+0000 7f7e8f400640 10 client.25609 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.072+0000 7f7e8f400640 1 -- 172.21.15.50:0/2066517681 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cc1203b80 con 0x558cc131b620 2024-09-12T03:31:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.072+0000 7f7e8f400640 10 client.25609 handle_command_reply: tid=0 2024-09-12T03:31:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.072+0000 7f7eb764a740 1 client.25609 shutdown 2024-09-12T03:31:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 msgr2=0x558cc12a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12a9ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8004740 tx=0x7f7ea80048e0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 shutdown_connections 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x558cc131b620 0x558cc131da40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cc12a13a0 0x558cc12a9ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cc1296b00 0x558cc12a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 --2- 172.21.15.50:0/2066517681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558cc12961e0 0x558cc129c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7e8f400640 0 client.25609 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 >> 172.21.15.50:0/2066517681 conn(0x558cc1292750 msgr2=0x558cc1291010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 shutdown_connections 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 1 -- 172.21.15.50:0/2066517681 wait complete. 2024-09-12T03:31:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.073+0000 7f7eb764a740 20 client.25609 trim_cache size 0 max 16384 2024-09-12T03:31:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:31:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:31:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:31:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:31:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7e940a2e20 msgr2=0x7f7e940a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7e940a2e20 0x7f7e940a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c00 tx=0x7f7ea0005b50 comp rx=0 tx=0).stop 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 msgr2=0x7f7eac19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 0x7f7eac19ab80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e9f0 tx=0x7f7e9c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 shutdown_connections 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eac19ef00 0x7f7eac19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eac1090a0 0x7f7eac19ab80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f7e940a2e20 0x7f7e940a52e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 --2- 172.21.15.50:0/3204544657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eac103f50 0x7f7eac19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 >> 172.21.15.50:0/3204544657 conn(0x7f7eac0ff9e0 msgr2=0x7f7eac101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 shutdown_connections 2024-09-12T03:31:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:31:56.074+0000 7f7eb4800640 1 -- 172.21.15.50:0/3204544657 wait complete. 2024-09-12T03:31:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:56 smithi050 ceph-mon[30378]: pgmap v146: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1023 KiB/s wr, 307 op/s 2024-09-12T03:31:57.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:56 smithi142 ceph-mon[35329]: pgmap v146: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1023 KiB/s wr, 307 op/s 2024-09-12T03:31:57.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:56 smithi082 ceph-mon[35046]: pgmap v146: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1023 KiB/s wr, 307 op/s 2024-09-12T03:31:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:31:58 smithi050 ceph-mon[30378]: pgmap v147: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-09-12T03:31:59.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:31:58 smithi142 ceph-mon[35329]: pgmap v147: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-09-12T03:31:59.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:31:58 smithi082 ceph-mon[35046]: pgmap v147: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.4 MiB/s wr, 491 op/s 2024-09-12T03:32:01.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:00 smithi142 ceph-mon[35329]: pgmap v148: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 995 KiB/s wr, 389 op/s 2024-09-12T03:32:01.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:00 smithi082 ceph-mon[35046]: pgmap v148: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 995 KiB/s wr, 389 op/s 2024-09-12T03:32:01.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:00 smithi050 ceph-mon[30378]: pgmap v148: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 995 KiB/s wr, 389 op/s 2024-09-12T03:32:03.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:02 smithi142 ceph-mon[35329]: pgmap v149: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 995 KiB/s wr, 389 op/s 2024-09-12T03:32:03.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:02 smithi082 ceph-mon[35046]: pgmap v149: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 995 KiB/s wr, 389 op/s 2024-09-12T03:32:03.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:02 smithi050 ceph-mon[30378]: pgmap v149: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 995 KiB/s wr, 389 op/s 2024-09-12T03:32:05.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:04 smithi142 ceph-mon[35329]: pgmap v150: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 604 op/s 2024-09-12T03:32:05.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:04 smithi082 ceph-mon[35046]: pgmap v150: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 604 op/s 2024-09-12T03:32:05.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:04 smithi050 ceph-mon[30378]: pgmap v150: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 604 op/s 2024-09-12T03:32:07.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:06 smithi142 ceph-mon[35329]: pgmap v151: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-09-12T03:32:07.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:06 smithi082 ceph-mon[35046]: pgmap v151: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-09-12T03:32:07.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:06 smithi050 ceph-mon[30378]: pgmap v151: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 -- 172.21.15.50:0/2843298838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 msgr2=0x7fb5d410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 --2- 172.21.15.50:0/2843298838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 0x7fb5d410dbb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009900 tx=0x7fb5c002a480 comp rx=0 tx=0).stop 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 -- 172.21.15.50:0/2843298838 shutdown_connections 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 --2- 172.21.15.50:0/2843298838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 0x7fb5d4110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 --2- 172.21.15.50:0/2843298838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 0x7fb5d410dbb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 --2- 172.21.15.50:0/2843298838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.234+0000 7fb5d9400640 1 -- 172.21.15.50:0/2843298838 >> 172.21.15.50:0/2843298838 conn(0x7fb5d40fd5d0 msgr2=0x7fb5d40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.235+0000 7fb5d9400640 1 -- 172.21.15.50:0/2843298838 shutdown_connections 2024-09-12T03:32:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.235+0000 7fb5d9400640 1 -- 172.21.15.50:0/2843298838 wait complete. 2024-09-12T03:32:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 Processor -- start 2024-09-12T03:32:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 -- start start 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 0x7fb5d419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 0x7fb5d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4113ea0 con 0x7fb5d410e0f0 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4113d20 con 0x7fb5d4074620 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d9400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4114020 con 0x7fb5d4074f40 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 0x7fb5d419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:48798/0 (socket says 172.21.15.50:48798) 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.236+0000 7fb5d3e00640 1 -- 172.21.15.50:0/3435435998 learned_addr learned my addr 172.21.15.50:0/3435435998 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 0x7fb5d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d3e00640 1 -- 172.21.15.50:0/3435435998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 msgr2=0x7fb5d419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d3e00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 0x7fb5d419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d3e00640 1 -- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 msgr2=0x7fb5d419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d3e00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 0x7fb5d419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d3e00640 1 -- 172.21.15.50:0/3435435998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d419f4a0 con 0x7fb5d4074620 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 0x7fb5d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.237+0000 7fb5d3e00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d419a4b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400ea60 tx=0x7fb5c400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.238+0000 7fb5d0c00640 1 -- 172.21.15.50:0/3435435998 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400cde0 con 0x7fb5d4074620 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.238+0000 7fb5d0c00640 1 -- 172.21.15.50:0/3435435998 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c40035f0 con 0x7fb5d4074620 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.238+0000 7fb5d0c00640 1 -- 172.21.15.50:0/3435435998 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4010670 con 0x7fb5d4074620 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.238+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d419f790 con 0x7fb5d4074620 2024-09-12T03:32:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.238+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41a7020 con 0x7fb5d4074620 2024-09-12T03:32:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 Processor -- start 2024-09-12T03:32:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5d0c00640 1 -- 172.21.15.50:0/3435435998 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb5c40037a0 con 0x7fb5d4074620 2024-09-12T03:32:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 -- start start 2024-09-12T03:32:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0cda960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c5a0c33100 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c5a0bdb1b0 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.243+0000 7fb5dd884740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c5a0c2f0c0 con 0x55c5a0cd8570 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:48800/0 (socket says 172.21.15.50:48800) 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2204478178 learned_addr learned my addr 172.21.15.50:0/2204478178 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d3400640 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0cda960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d3e00640 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2204478178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 msgr2=0x55c5a0cda960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0cda960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2204478178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 msgr2=0x55c5a0cd8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2204478178 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a0946210 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccd790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009fd0 tx=0x7fb5cc010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2da29d5d67b5e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5c9200640 1 -- 172.21.15.50:0/2204478178 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc011700 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5c9200640 1 -- 172.21.15.50:0/2204478178 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5cc011d80 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5c9200640 1 -- 172.21.15.50:0/2204478178 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00e3c0 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.244+0000 7fb5d0c00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb5b80a2cf0 0x7fb5b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.245+0000 7fb5d3e00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb5b80a2cf0 0x7fb5b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.245+0000 7fb5d0c00640 1 -- 172.21.15.50:0/3435435998 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fb5c40cfbc0 con 0x7fb5d4074620 2024-09-12T03:32:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.245+0000 7fb5d3e00640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb5b80a2cf0 0x7fb5b80a51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55c5a0cca780 tx=0x7fb5c4003f30 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.245+0000 7fb5dd884740 1 -- 172.21.15.50:0/2204478178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 msgr2=0x55c5a0ccd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.245+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccd790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009fd0 tx=0x7fb5cc010860 comp rx=0 tx=0).stop 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 -- 172.21.15.50:0/2204478178 shutdown_connections 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0cda960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2204478178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccd790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 -- 172.21.15.50:0/2204478178 >> 172.21.15.50:0/2204478178 conn(0x55c5a0cc9920 msgr2=0x55c5a0cc9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 -- 172.21.15.50:0/2204478178 shutdown_connections 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 -- 172.21.15.50:0/2204478178 wait complete. 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 -- start start 2024-09-12T03:32:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.246+0000 7fb5dd884740 1 Processor -- start 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cd5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0cd6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c5a0cdaf90 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c5a0bbc090 con 0x55c5a0cd8570 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c5a0cdb110 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.247+0000 7fb5dd884740 10 client.0 _ll_register_callbacks cb 0x55c5a0c10f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.248+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cd5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.248+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cd5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42736/0 (socket says 172.21.15.50:42736) 2024-09-12T03:32:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.248+0000 7fb5d8a00640 1 -- 172.21.15.50:0/3014114834 learned_addr learned my addr 172.21.15.50:0/3014114834 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.248+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0cd6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.249+0000 7fb5d8a00640 1 -- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 msgr2=0x55c5a0cd6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.249+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0cd6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.249+0000 7fb5d8a00640 1 -- 172.21.15.50:0/3014114834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 msgr2=0x55c5a0cd5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.249+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.249+0000 7fb5d8a00640 1 -- 172.21.15.50:0/3014114834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a0a12620 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.250+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cd5690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc011fd0 tx=0x7fb5cc01bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c345a61d1629480 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.250+0000 7fb5bd600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.250+0000 7fb5bd600640 1 -- 172.21.15.50:0/3014114834 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc034060 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.251+0000 7fb5bd600640 1 -- 172.21.15.50:0/3014114834 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fb5cc011700 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.251+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5cc011d80 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.251+0000 7fb5bd600640 1 -- 172.21.15.50:0/3014114834 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb5cc03b050 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.251+0000 7fb5dd884740 10 client.25621 fetch_fsmap learned FSMap version 30 2024-09-12T03:32:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.251+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5a0cf7e50 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.252+0000 7fb5bd600640 1 -- 172.21.15.50:0/3014114834 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5cc011900 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5bd600640 1 -- 172.21.15.50:0/3014114834 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb5cc00eb00 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5dd884740 10 client.25621 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:32:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5dd884740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5dd884740 20 client.25621 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5dd884740 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0cf9310 0x55c5a0cfb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5dd884740 4 client.25621 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.253+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb5cc011d80 con 0x55c5a0cf9310 2024-09-12T03:32:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.254+0000 7fb5d3400640 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0cf9310 0x55c5a0cfb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.256+0000 7fb5d3400640 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0cf9310 0x55c5a0cfb700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ebf21f211c3a28 server_cookie=b57d79faabb37ae9 in_seq=0 out_seq=0 2024-09-12T03:32:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.256+0000 7fb5bd600640 10 client.25621 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.257+0000 7fb5bd600640 1 -- 172.21.15.50:0/3014114834 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb5cc011d80 con 0x55c5a0cf9310 2024-09-12T03:32:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.257+0000 7fb5bd600640 10 client.25621 handle_command_reply: tid=0 2024-09-12T03:32:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.257+0000 7fb5dd884740 1 client.25621 shutdown 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.259+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 msgr2=0x55c5a0cd5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.259+0000 7fb5dd884740 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cd5690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc011fd0 tx=0x7fb5cc01bcc0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 shutdown_connections 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0cd6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0cf9310 0x55c5a0cfb700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 --2- 172.21.15.50:0/3014114834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cd5690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 >> 172.21.15.50:0/3014114834 conn(0x55c5a0cc9920 msgr2=0x55c5a0ccf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5bd600640 0 client.25621 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 shutdown_connections 2024-09-12T03:32:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 1 -- 172.21.15.50:0/3014114834 wait complete. 2024-09-12T03:32:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.260+0000 7fb5dd884740 20 client.25621 trim_cache size 0 max 16384 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 Processor -- start 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 -- start start 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0d00e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4074050 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40718c0 con 0x55c5a0cd8570 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.280+0000 7fb5dd884740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d406f710 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33220/0 (socket says 172.21.15.50:33220) 2024-09-12T03:32:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2025762687 learned_addr learned my addr 172.21.15.50:0/2025762687 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d3400640 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0d00e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2025762687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccdcd0 msgr2=0x55c5a0cd74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2025762687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 msgr2=0x55c5a0d00e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0d00e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 -- 172.21.15.50:0/2025762687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a0cdacd0 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d3400640 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.281+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccfe20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006010 tx=0x7fb5cc009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d99905c0365fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.282+0000 7fb5bfe00640 1 -- 172.21.15.50:0/2025762687 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc024040 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.282+0000 7fb5bfe00640 1 -- 172.21.15.50:0/2025762687 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5cc019790 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.282+0000 7fb5bfe00640 1 -- 172.21.15.50:0/2025762687 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc011760 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 -- 172.21.15.50:0/2025762687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 msgr2=0x55c5a0ccfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccfe20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006010 tx=0x7fb5cc009610 comp rx=0 tx=0).stop 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 -- 172.21.15.50:0/2025762687 shutdown_connections 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0cd8570 0x55c5a0d00e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cd74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 --2- 172.21.15.50:0/2025762687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0ccfe20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 -- 172.21.15.50:0/2025762687 >> 172.21.15.50:0/2025762687 conn(0x55c5a0cc9920 msgr2=0x55c5a0cd6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 -- 172.21.15.50:0/2025762687 shutdown_connections 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.283+0000 7fb5dd884740 1 -- 172.21.15.50:0/2025762687 wait complete. 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 -- start start 2024-09-12T03:32:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 Processor -- start 2024-09-12T03:32:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cdbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cdc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0d5bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d406fd80 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40711f0 con 0x55c5a0ccd3b0 2024-09-12T03:32:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4070450 con 0x55c5a0cd8570 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.284+0000 7fb5dd884740 10 client.0 _ll_register_callbacks cb 0x7fb5c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cdc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cdbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cdbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:48828/0 (socket says 172.21.15.50:48828) 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cdc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33222/0 (socket says 172.21.15.50:33222) 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d8a00640 1 -- 172.21.15.50:0/448908755 learned_addr learned my addr 172.21.15.50:0/448908755 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0d5bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 -- 172.21.15.50:0/448908755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 msgr2=0x55c5a0d5bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0d5bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 -- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 msgr2=0x55c5a0cdbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cdbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 -- 172.21.15.50:0/448908755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a0d5c450 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cdbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5d3400640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cdc0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0005940 tx=0x7fb5c0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7710ebac137ee618 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.285+0000 7fb5bd600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.286+0000 7fb5bd600640 1 -- 172.21.15.50:0/448908755 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0049070 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.286+0000 7fb5bd600640 1 -- 172.21.15.50:0/448908755 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fb5c00317b0 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.286+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5cc019790 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5bd600640 1 -- 172.21.15.50:0/448908755 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb5c0054050 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 10 client.15603 fetch_fsmap learned FSMap version 30 2024-09-12T03:32:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5a0d63820 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5bd600640 1 -- 172.21.15.50:0/448908755 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5cc019790 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5bd600640 1 -- 172.21.15.50:0/448908755 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb5c0045360 con 0x55c5a0ccdcd0 2024-09-12T03:32:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 10 client.15603 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:32:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 20 client.15603 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0d63f40 0x55c5a0d66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 4 client.15603 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.287+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5a0c40830 con 0x55c5a0d63f40 2024-09-12T03:32:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.288+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0d63f40 0x55c5a0d66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.288+0000 7fb5d8a00640 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0d63f40 0x55c5a0d66360 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b841269c99a50fd server_cookie=3bf471f76aff31a in_seq=0 out_seq=0 2024-09-12T03:32:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.288+0000 7fb5bd600640 10 client.15603 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:09.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:08 smithi142 ceph-mon[35329]: pgmap v152: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.301+0000 7fb5bd600640 1 -- 172.21.15.50:0/448908755 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5a0c40830 con 0x55c5a0d63f40 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.301+0000 7fb5bd600640 10 client.15603 handle_command_reply: tid=0 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.301+0000 7fb5dd884740 1 client.15603 shutdown 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.301+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 msgr2=0x55c5a0cdc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cdc0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0005940 tx=0x7fb5c0004290 comp rx=0 tx=0).stop 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 shutdown_connections 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55c5a0d63f40 0x55c5a0d66360 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5a0cd8570 0x55c5a0d5bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5a0ccdcd0 0x55c5a0cdc0a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5bd600640 0 client.15603 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 --2- 172.21.15.50:0/448908755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5a0ccd3b0 0x55c5a0cdbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 >> 172.21.15.50:0/448908755 conn(0x55c5a0cc9920 msgr2=0x55c5a0cc89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 shutdown_connections 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 1 -- 172.21.15.50:0/448908755 wait complete. 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.302+0000 7fb5dd884740 20 client.15603 trim_cache size 0 max 16384 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:32:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:32:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb5b80a2cf0 msgr2=0x7fb5b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb5b80a2cf0 0x7fb5b80a51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55c5a0cca780 tx=0x7fb5c4003f30 comp rx=0 tx=0).stop 2024-09-12T03:32:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 msgr2=0x7fb5d419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d419a4b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400ea60 tx=0x7fb5c400ef20 comp rx=0 tx=0).stop 2024-09-12T03:32:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 shutdown_connections 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb5b80a2cf0 0x7fb5b80a51b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5d410e0f0 0x7fb5d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4074f40 0x7fb5d419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 --2- 172.21.15.50:0/3435435998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4074620 0x7fb5d419a4b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 >> 172.21.15.50:0/3435435998 conn(0x7fb5d40fd5d0 msgr2=0x7fb5d40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 shutdown_connections 2024-09-12T03:32:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:09.303+0000 7fb5d9400640 1 -- 172.21.15.50:0/3435435998 wait complete. 2024-09-12T03:32:09.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:08 smithi082 ceph-mon[35046]: pgmap v152: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-09-12T03:32:09.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:08 smithi050 ceph-mon[30378]: pgmap v152: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-09-12T03:32:09.493 INFO:tasks.workunit.client.0.smithi050.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-09-12T03:32:11.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:10 smithi142 ceph-mon[35329]: pgmap v153: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:32:11.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:10 smithi082 ceph-mon[35046]: pgmap v153: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:32:11.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:10 smithi050 ceph-mon[30378]: pgmap v153: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.3 MiB/s wr, 417 op/s 2024-09-12T03:32:13.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:12 smithi142 ceph-mon[35329]: pgmap v154: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-09-12T03:32:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:12 smithi082 ceph-mon[35046]: pgmap v154: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-09-12T03:32:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:12 smithi050 ceph-mon[30378]: pgmap v154: 129 pgs: 129 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-09-12T03:32:15.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:14 smithi142 ceph-mon[35329]: pgmap v155: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.7 MiB/s wr, 590 op/s 2024-09-12T03:32:15.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:14 smithi082 ceph-mon[35046]: pgmap v155: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.7 MiB/s wr, 590 op/s 2024-09-12T03:32:15.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:14 smithi050 ceph-mon[30378]: pgmap v155: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.7 MiB/s wr, 590 op/s 2024-09-12T03:32:17.217 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:16 smithi050 ceph-mon[30378]: pgmap v156: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.2 MiB/s wr, 375 op/s 2024-09-12T03:32:17.217 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:16 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:32:17.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:16 smithi142 ceph-mon[35329]: pgmap v156: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.2 MiB/s wr, 375 op/s 2024-09-12T03:32:17.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:16 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:32:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:16 smithi082 ceph-mon[35046]: pgmap v156: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.2 MiB/s wr, 375 op/s 2024-09-12T03:32:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:16 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:32:19.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:18 smithi142 ceph-mon[35329]: pgmap v157: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 565 op/s 2024-09-12T03:32:19.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:18 smithi082 ceph-mon[35046]: pgmap v157: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 565 op/s 2024-09-12T03:32:19.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:18 smithi050 ceph-mon[30378]: pgmap v157: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 565 op/s 2024-09-12T03:32:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:20 smithi142 ceph-mon[35329]: pgmap v158: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.1 MiB/s wr, 411 op/s 2024-09-12T03:32:21.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:20 smithi082 ceph-mon[35046]: pgmap v158: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.1 MiB/s wr, 411 op/s 2024-09-12T03:32:21.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:20 smithi050 ceph-mon[30378]: pgmap v158: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.1 MiB/s wr, 411 op/s 2024-09-12T03:32:22.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:21 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:22.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:21 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:22.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:21 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.459+0000 7f0c68400640 1 -- 172.21.15.50:0/2912585383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 msgr2=0x7f0c601095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.459+0000 7f0c68400640 1 --2- 172.21.15.50:0/2912585383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601095d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009900 tx=0x7f0c5402a400 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.460+0000 7f0c68400640 1 -- 172.21.15.50:0/2912585383 shutdown_connections 2024-09-12T03:32:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.461+0000 7f0c68400640 1 --2- 172.21.15.50:0/2912585383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c60116ea0 0x7f0c60119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.461+0000 7f0c68400640 1 --2- 172.21.15.50:0/2912585383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60109b10 0x7f0c60116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.461+0000 7f0c68400640 1 --2- 172.21.15.50:0/2912585383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601095d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.461+0000 7f0c68400640 1 -- 172.21.15.50:0/2912585383 >> 172.21.15.50:0/2912585383 conn(0x7f0c6006ce00 msgr2=0x7f0c6006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.462+0000 7f0c68400640 1 -- 172.21.15.50:0/2912585383 shutdown_connections 2024-09-12T03:32:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.462+0000 7f0c68400640 1 -- 172.21.15.50:0/2912585383 wait complete. 2024-09-12T03:32:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 Processor -- start 2024-09-12T03:32:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 -- start start 2024-09-12T03:32:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c60109b10 0x7f0c601a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60116ea0 0x7f0c601a7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6011c9b0 con 0x7f0c60116ea0 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6011c830 con 0x7f0c601091f0 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.464+0000 7f0c68400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6011cb30 con 0x7f0c60109b10 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39022/0 (socket says 172.21.15.50:39022) 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 -- 172.21.15.50:0/496173118 learned_addr learned my addr 172.21.15.50:0/496173118 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65000640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c60109b10 0x7f0c601a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c66400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60116ea0 0x7f0c601a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 -- 172.21.15.50:0/496173118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c60109b10 msgr2=0x7f0c601a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c60109b10 0x7f0c601a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 -- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60116ea0 msgr2=0x7f0c601a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60116ea0 0x7f0c601a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 -- 172.21.15.50:0/496173118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601a7da0 con 0x7f0c601091f0 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c66400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60116ea0 0x7f0c601a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.465+0000 7f0c65a00640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601a2db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c54004770 tx=0x7f0c54003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.466+0000 7f0c5e000640 1 -- 172.21.15.50:0/496173118 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5403f070 con 0x7f0c601091f0 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.466+0000 7f0c5e000640 1 -- 172.21.15.50:0/496173118 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c54031ca0 con 0x7f0c601091f0 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.466+0000 7f0c5e000640 1 -- 172.21.15.50:0/496173118 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54035030 con 0x7f0c601091f0 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.466+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601a8030 con 0x7f0c601091f0 2024-09-12T03:32:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.466+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c601af920 con 0x7f0c601091f0 2024-09-12T03:32:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.471+0000 7f0c5e000640 1 -- 172.21.15.50:0/496173118 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0c54042070 con 0x7f0c601091f0 2024-09-12T03:32:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.471+0000 7f0c6b288740 1 Processor -- start 2024-09-12T03:32:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 -- start start 2024-09-12T03:32:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e3be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e3e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ca0d97100 con 0x563ca0e311e0 2024-09-12T03:32:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563ca0d3f1b0 con 0x563ca0e3c3a0 2024-09-12T03:32:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.472+0000 7f0c6b288740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ca0d930c0 con 0x563ca0e31b00 2024-09-12T03:32:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e3be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e3e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39028/0 (socket says 172.21.15.50:39028) 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e3be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36212/0 (socket says 172.21.15.50:36212) 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65000640 1 -- 172.21.15.50:0/2081976423 learned_addr learned my addr 172.21.15.50:0/2081976423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c66400640 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c5e000640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0c480a2e20 0x7f0c480a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c5e000640 1 -- 172.21.15.50:0/496173118 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f0c540eff60 con 0x7f0c601091f0 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 -- 172.21.15.50:0/2081976423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 msgr2=0x563ca0e3e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e3e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 -- 172.21.15.50:0/2081976423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 msgr2=0x563ca0e315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c65a00640 1 -- 172.21.15.50:0/2081976423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca0aaa210 con 0x563ca0e31b00 2024-09-12T03:32:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c66400640 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.473+0000 7f0c66400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0c480a2e20 0x7f0c480a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.474+0000 7f0c65a00640 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e3be60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c540ed320 tx=0x7f0c5403b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c87ac5eaca2eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.474+0000 7f0c52a00640 1 -- 172.21.15.50:0/2081976423 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54049070 con 0x563ca0e31b00 2024-09-12T03:32:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.474+0000 7f0c52a00640 1 -- 172.21.15.50:0/2081976423 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c540c04a0 con 0x563ca0e31b00 2024-09-12T03:32:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.474+0000 7f0c52a00640 1 -- 172.21.15.50:0/2081976423 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540b8760 con 0x563ca0e31b00 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 -- 172.21.15.50:0/2081976423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 msgr2=0x563ca0e3be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e3be60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c540ed320 tx=0x7f0c5403b460 comp rx=0 tx=0).stop 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c66400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0c480a2e20 0x7f0c480a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c58005c40 tx=0x7f0c58004a40 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 -- 172.21.15.50:0/2081976423 shutdown_connections 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e3e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e3be60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2081976423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 -- 172.21.15.50:0/2081976423 >> 172.21.15.50:0/2081976423 conn(0x563ca0e2d750 msgr2=0x563ca0e2db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 -- 172.21.15.50:0/2081976423 shutdown_connections 2024-09-12T03:32:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.475+0000 7f0c6b288740 1 -- 172.21.15.50:0/2081976423 wait complete. 2024-09-12T03:32:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.476+0000 7f0c6b288740 1 -- start start 2024-09-12T03:32:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 Processor -- start 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e31b00 0x563ca0e39920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 0x563ca0e39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ca0e3ed30 con 0x563ca0e311e0 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563ca0d20090 con 0x563ca0e31b00 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ca0e3eeb0 con 0x563ca0e3c3a0 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c6b288740 10 client.0 _ll_register_callbacks cb 0x563ca0d74f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c66400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 0x563ca0e39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.477+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 0x563ca0e39e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:36220/0 (socket says 172.21.15.50:36220) 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 -- 172.21.15.50:0/3466340402 learned_addr learned my addr 172.21.15.50:0/3466340402 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c65000640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e31b00 0x563ca0e39920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 -- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e31b00 msgr2=0x563ca0e39920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e31b00 0x563ca0e39920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 -- 172.21.15.50:0/3466340402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 msgr2=0x563ca0e393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 -- 172.21.15.50:0/3466340402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca0b76620 con 0x563ca0e3c3a0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c65000640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e31b00 0x563ca0e39920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c66400640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 0x563ca0e39e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009c90 tx=0x7f0c58004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ea2f3aabde36c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.478+0000 7f0c47400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c47400640 1 -- 172.21.15.50:0/3466340402 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5801ca30 con 0x563ca0e3c3a0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c47400640 1 -- 172.21.15.50:0/3466340402 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f0c5801cbd0 con 0x563ca0e3c3a0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c540c04a0 con 0x563ca0e3c3a0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c47400640 1 -- 172.21.15.50:0/3466340402 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0c58030050 con 0x563ca0e3c3a0 2024-09-12T03:32:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c6b288740 10 client.25636 fetch_fsmap learned FSMap version 31 2024-09-12T03:32:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563ca0e53200 con 0x563ca0e3c3a0 2024-09-12T03:32:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.479+0000 7f0c47400640 1 -- 172.21.15.50:0/3466340402 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c540c04a0 con 0x563ca0e3c3a0 2024-09-12T03:32:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c47400640 1 -- 172.21.15.50:0/3466340402 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f0c5801a4a0 con 0x563ca0e3c3a0 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c6b288740 10 client.25636 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c6b288740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c6b288740 20 client.25636 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c6b288740 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0e54780 0x563ca0e56bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c6b288740 4 client.25636 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca0da4830 con 0x563ca0e54780 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.480+0000 7f0c65a00640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0e54780 0x563ca0e56bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.481+0000 7f0c65a00640 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0e54780 0x563ca0e56bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e2175ba54ece528 server_cookie=4ba19ffc9197c8aa in_seq=0 out_seq=0 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.481+0000 7f0c47400640 10 client.25636 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.481+0000 7f0c47400640 1 -- 172.21.15.50:0/3466340402 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ca0da4830 con 0x563ca0e54780 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.481+0000 7f0c47400640 10 client.25636 handle_command_reply: tid=0 2024-09-12T03:32:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.481+0000 7f0c6b288740 1 client.25636 shutdown 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 msgr2=0x563ca0e39e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 0x563ca0e39e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009c90 tx=0x7f0c58004ab0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 shutdown_connections 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0e54780 0x563ca0e56bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e3c3a0 0x563ca0e39e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e31b00 0x563ca0e39920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 --2- 172.21.15.50:0/3466340402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e311e0 0x563ca0e393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c47400640 0 client.25636 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 >> 172.21.15.50:0/3466340402 conn(0x563ca0e2d750 msgr2=0x563ca0e334b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 shutdown_connections 2024-09-12T03:32:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 1 -- 172.21.15.50:0/3466340402 wait complete. 2024-09-12T03:32:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.482+0000 7f0c6b288740 20 client.25636 trim_cache size 0 max 16384 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 Processor -- start 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 -- start start 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e311e0 0x563ca0e5c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e31b00 0x563ca0e5ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601b7ec0 con 0x563ca0e31b00 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601b5760 con 0x563ca0e3c3a0 2024-09-12T03:32:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c6b288740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601b35f0 con 0x563ca0e311e0 2024-09-12T03:32:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.498+0000 7f0c65000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e31b00 0x563ca0e5ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c65000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e31b00 0x563ca0e5ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52770/0 (socket says 172.21.15.50:52770) 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c65000640 1 -- 172.21.15.50:0/1184434394 learned_addr learned my addr 172.21.15.50:0/1184434394 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c65a00640 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e311e0 0x563ca0e5c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 -- 172.21.15.50:0/1184434394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e311e0 msgr2=0x563ca0e5c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e311e0 0x563ca0e5c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 -- 172.21.15.50:0/1184434394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e31b00 msgr2=0x563ca0e5ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e31b00 0x563ca0e5ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 -- 172.21.15.50:0/1184434394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca0e31010 con 0x563ca0e3c3a0 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.499+0000 7f0c66400640 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36640 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009c60 tx=0x7f0c58005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e816ef44d3e846 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.500+0000 7f0c52000640 1 -- 172.21.15.50:0/1184434394 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58038070 con 0x563ca0e3c3a0 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.500+0000 7f0c52000640 1 -- 172.21.15.50:0/1184434394 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5801cd10 con 0x563ca0e3c3a0 2024-09-12T03:32:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.500+0000 7f0c52000640 1 -- 172.21.15.50:0/1184434394 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58034430 con 0x563ca0e3c3a0 2024-09-12T03:32:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 -- 172.21.15.50:0/1184434394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 msgr2=0x563ca0e36640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36640 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009c60 tx=0x7f0c58005fd0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 -- 172.21.15.50:0/1184434394 shutdown_connections 2024-09-12T03:32:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36640 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e31b00 0x563ca0e5ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 --2- 172.21.15.50:0/1184434394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e311e0 0x563ca0e5c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 -- 172.21.15.50:0/1184434394 >> 172.21.15.50:0/1184434394 conn(0x563ca0e2d750 msgr2=0x563ca0e346c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 -- 172.21.15.50:0/1184434394 shutdown_connections 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 -- 172.21.15.50:0/1184434394 wait complete. 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.501+0000 7f0c6b288740 1 -- start start 2024-09-12T03:32:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 Processor -- start 2024-09-12T03:32:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 0x563ca0e36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601b3c40 con 0x563ca0e3c3a0 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601b5090 con 0x563ca0e311e0 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601b42f0 con 0x563ca0e31b00 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.502+0000 7f0c6b288740 10 client.0 _ll_register_callbacks cb 0x7f0c580097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 0x563ca0e36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 0x563ca0e36340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39060/0 (socket says 172.21.15.50:39060) 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 -- 172.21.15.50:0/2403511642 learned_addr learned my addr 172.21.15.50:0/2403511642 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c66400640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65000640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 -- 172.21.15.50:0/2403511642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 msgr2=0x563ca0e36880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e36880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 -- 172.21.15.50:0/2403511642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e3c3a0 msgr2=0x563ca0e36dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 -- 172.21.15.50:0/2403511642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca0e2e3e0 con 0x563ca0e311e0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c66400640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c65a00640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 0x563ca0e36340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c540b8ac0 tx=0x7f0c54009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a249bbaaa47cca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c47400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.503+0000 7f0c47400640 1 -- 172.21.15.50:0/2403511642 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54049040 con 0x563ca0e311e0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c47400640 1 -- 172.21.15.50:0/2403511642 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f0c540c0160 con 0x563ca0e311e0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5801cd10 con 0x563ca0e311e0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c47400640 1 -- 172.21.15.50:0/2403511642 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0c540c04a0 con 0x563ca0e311e0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 10 client.25652 fetch_fsmap learned FSMap version 31 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563ca0e28d60 con 0x563ca0e311e0 2024-09-12T03:32:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c47400640 1 -- 172.21.15.50:0/2403511642 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5801cd10 con 0x563ca0e311e0 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c47400640 1 -- 172.21.15.50:0/2403511642 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f0c540d6070 con 0x563ca0e311e0 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 10 client.25652 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 20 client.25652 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0ebf840 0x563ca0ec1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 4 client.25652 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.504+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ca0d9eb80 con 0x563ca0ebf840 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.505+0000 7f0c65000640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0ebf840 0x563ca0ec1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.505+0000 7f0c65000640 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0ebf840 0x563ca0ec1c60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3dce712a83db0f server_cookie=3b92dbec5efe5845 in_seq=0 out_seq=0 2024-09-12T03:32:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.505+0000 7f0c47400640 10 client.25652 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.528+0000 7f0c47400640 1 -- 172.21.15.50:0/2403511642 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ca0d9eb80 con 0x563ca0ebf840 2024-09-12T03:32:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.528+0000 7f0c47400640 10 client.25652 handle_command_reply: tid=0 2024-09-12T03:32:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.528+0000 7f0c6b288740 1 client.25652 shutdown 2024-09-12T03:32:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 msgr2=0x563ca0e36340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 0x563ca0e36340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c540b8ac0 tx=0x7f0c54009680 comp rx=0 tx=0).stop 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 shutdown_connections 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ca0e3c3a0 0x563ca0e36dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x563ca0ebf840 0x563ca0ec1c60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca0e31b00 0x563ca0e36880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 --2- 172.21.15.50:0/2403511642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ca0e311e0 0x563ca0e36340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c47400640 0 client.25652 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 >> 172.21.15.50:0/2403511642 conn(0x563ca0e2d750 msgr2=0x563ca0e393a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 shutdown_connections 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 1 -- 172.21.15.50:0/2403511642 wait complete. 2024-09-12T03:32:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.529+0000 7f0c6b288740 20 client.25652 trim_cache size 0 max 16384 2024-09-12T03:32:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:32:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:32:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:32:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0c480a2e20 msgr2=0x7f0c480a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0c480a2e20 0x7f0c480a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c58005c40 tx=0x7f0c58004a40 comp rx=0 tx=0).stop 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 msgr2=0x7f0c601a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601a2db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c54004770 tx=0x7f0c54003c10 comp rx=0 tx=0).stop 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 shutdown_connections 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0c480a2e20 0x7f0c480a52e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c60116ea0 0x7f0c601a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c60109b10 0x7f0c601a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 --2- 172.21.15.50:0/496173118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c601091f0 0x7f0c601a2db0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 >> 172.21.15.50:0/496173118 conn(0x7f0c6006ce00 msgr2=0x7f0c60117320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 shutdown_connections 2024-09-12T03:32:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:22.530+0000 7f0c68400640 1 -- 172.21.15.50:0/496173118 wait complete. 2024-09-12T03:32:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:22 smithi142 ceph-mon[35329]: pgmap v159: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.1 MiB/s wr, 394 op/s 2024-09-12T03:32:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:22 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:22 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:22 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:32:23.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:22 smithi082 ceph-mon[35046]: pgmap v159: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.1 MiB/s wr, 394 op/s 2024-09-12T03:32:23.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:22 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:23.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:22 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:23.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:22 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:32:23.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:22 smithi050 ceph-mon[30378]: pgmap v159: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.1 MiB/s wr, 394 op/s 2024-09-12T03:32:23.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:22 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:23.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:22 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:23.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:22 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:32:24.408 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x63e5d2a 2024-09-12T03:32:25.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:24 smithi142 ceph-mon[35329]: pgmap v160: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.7 MiB/s wr, 584 op/s 2024-09-12T03:32:25.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:24 smithi082 ceph-mon[35046]: pgmap v160: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.7 MiB/s wr, 584 op/s 2024-09-12T03:32:25.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:24 smithi050 ceph-mon[30378]: pgmap v160: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.7 MiB/s wr, 584 op/s 2024-09-12T03:32:27.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:26 smithi142 ceph-mon[35329]: pgmap v161: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-09-12T03:32:27.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:26 smithi082 ceph-mon[35046]: pgmap v161: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-09-12T03:32:27.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:26 smithi050 ceph-mon[30378]: pgmap v161: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-09-12T03:32:29.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:28 smithi142 ceph-mon[35329]: pgmap v162: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 571 op/s 2024-09-12T03:32:29.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:28 smithi082 ceph-mon[35046]: pgmap v162: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 571 op/s 2024-09-12T03:32:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:28 smithi050 ceph-mon[30378]: pgmap v162: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 571 op/s 2024-09-12T03:32:30.118 INFO:tasks.workunit.client.0.smithi050.stdout:truncating to largest ever: 0x63e9b52 2024-09-12T03:32:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:30 smithi142 ceph-mon[35329]: pgmap v163: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 397 op/s 2024-09-12T03:32:31.356 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:30 smithi082 ceph-mon[35046]: pgmap v163: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 397 op/s 2024-09-12T03:32:31.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:30 smithi050 ceph-mon[30378]: pgmap v163: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 397 op/s 2024-09-12T03:32:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:32 smithi142 ceph-mon[35329]: pgmap v164: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-09-12T03:32:33.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:32 smithi082 ceph-mon[35046]: pgmap v164: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-09-12T03:32:33.416 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:32 smithi050 ceph-mon[30378]: pgmap v164: 129 pgs: 129 active+clean; 97 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-09-12T03:32:34.719 INFO:tasks.workunit.client.0.smithi050.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-09-12T03:32:35.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:34 smithi142 ceph-mon[35329]: pgmap v165: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 562 op/s 2024-09-12T03:32:35.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:34 smithi082 ceph-mon[35046]: pgmap v165: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 562 op/s 2024-09-12T03:32:36.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:34 smithi050 ceph-mon[30378]: pgmap v165: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 562 op/s 2024-09-12T03:32:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.671+0000 7f312f600640 1 -- 172.21.15.50:0/3528852368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 msgr2=0x7f3128107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.671+0000 7f312f600640 1 --2- 172.21.15.50:0/3528852368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f3128107f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f311c009890 tx=0x7f311c02a380 comp rx=0 tx=0).stop 2024-09-12T03:32:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 -- 172.21.15.50:0/3528852368 shutdown_connections 2024-09-12T03:32:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 --2- 172.21.15.50:0/3528852368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 --2- 172.21.15.50:0/3528852368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 0x7f3128101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 --2- 172.21.15.50:0/3528852368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f3128107f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 -- 172.21.15.50:0/3528852368 >> 172.21.15.50:0/3528852368 conn(0x7f31280fd7d0 msgr2=0x7f31280ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 -- 172.21.15.50:0/3528852368 shutdown_connections 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.672+0000 7f312f600640 1 -- 172.21.15.50:0/3528852368 wait complete. 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.673+0000 7f312f600640 1 Processor -- start 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.673+0000 7f312f600640 1 -- start start 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312f600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 0x7f312819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312f600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f312819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312f600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3128111ee0 con 0x7f3128101ad0 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312f600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3128111d60 con 0x7f3128102470 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312f600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3128112060 con 0x7f3128107b70 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f3127e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f3127e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39676/0 (socket says 172.21.15.50:39676) 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f3127e00640 1 -- 172.21.15.50:0/3166060780 learned_addr learned my addr 172.21.15.50:0/3166060780 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.674+0000 7f312cc00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 0x7f312819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f312d600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f312819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3127e00640 1 -- 172.21.15.50:0/3166060780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 msgr2=0x7f312819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3127e00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f312819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3127e00640 1 -- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 msgr2=0x7f312819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3127e00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 0x7f312819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3127e00640 1 -- 172.21.15.50:0/3166060780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312819f550 con 0x7f3128102470 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f312d600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f312819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f312cc00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 0x7f312819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3127e00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312819aaa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f311400d950 tx=0x7f311400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.675+0000 7f3125600640 1 -- 172.21.15.50:0/3166060780 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114017070 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.676+0000 7f3125600640 1 -- 172.21.15.50:0/3166060780 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3114009c30 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.676+0000 7f3125600640 1 -- 172.21.15.50:0/3166060780 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311400f690 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.676+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312819f840 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.676+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3128108930 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.680+0000 7f3125600640 1 -- 172.21.15.50:0/3166060780 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f31140109d0 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.681+0000 7f3125600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f31100a2e20 0x7f31100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.681+0000 7f3132666740 1 Processor -- start 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3125600640 1 -- 172.21.15.50:0/3166060780 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f31140cf4a0 con 0x7f3128102470 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 -- start start 2024-09-12T03:32:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f312cc00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f31100a2e20 0x7f31100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630be05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630beae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630bed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f630b46100 con 0x55f630be01e0 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f630aee1b0 con 0x55f630beb3a0 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f3132666740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f630b420c0 con 0x55f630be0b00 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.682+0000 7f312d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630bed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f312d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630bed790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39678/0 (socket says 172.21.15.50:39678) 2024-09-12T03:32:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f312d600640 1 -- 172.21.15.50:0/3391005500 learned_addr learned my addr 172.21.15.50:0/3391005500 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630beae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f312cc00640 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630be05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f312cc00640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f31100a2e20 0x7f31100a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f311c000c00 tx=0x7f311c005d60 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 -- 172.21.15.50:0/3391005500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 msgr2=0x55f630bed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630bed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 -- 172.21.15.50:0/3391005500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 msgr2=0x55f630be05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630be05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 -- 172.21.15.50:0/3391005500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f630859210 con 0x55f630be0b00 2024-09-12T03:32:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.683+0000 7f3127e00640 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630beae60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31140cc860 tx=0x7f311409bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e45352b4a55131 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.684+0000 7f3119c00640 1 -- 172.21.15.50:0/3391005500 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114020040 con 0x55f630be0b00 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.684+0000 7f3119c00640 1 -- 172.21.15.50:0/3391005500 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31140a42a0 con 0x55f630be0b00 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.684+0000 7f3119c00640 1 -- 172.21.15.50:0/3391005500 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114097ca0 con 0x55f630be0b00 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 -- 172.21.15.50:0/3391005500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 msgr2=0x55f630beae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630beae60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31140cc860 tx=0x7f311409bdf0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 -- 172.21.15.50:0/3391005500 shutdown_connections 2024-09-12T03:32:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630bed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630beae60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 --2- 172.21.15.50:0/3391005500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630be05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 -- 172.21.15.50:0/3391005500 >> 172.21.15.50:0/3391005500 conn(0x55f630bdc750 msgr2=0x55f630bdcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 -- 172.21.15.50:0/3391005500 shutdown_connections 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 -- 172.21.15.50:0/3391005500 wait complete. 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.685+0000 7f3132666740 1 -- start start 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 Processor -- start 2024-09-12T03:32:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be01e0 0x55f630be83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 0x55f630be88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630be8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f630bedf10 con 0x55f630beb3a0 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f630acf090 con 0x55f630be0b00 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f630bee090 con 0x55f630be01e0 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3132666740 10 client.0 _ll_register_callbacks cb 0x55f630b23f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f312d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630be8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3127e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 0x55f630be88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f3127e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 0x55f630be88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:39694/0 (socket says 172.21.15.50:39694) 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f312d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630be8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43708/0 (socket says 172.21.15.50:43708) 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f312d600640 1 -- 172.21.15.50:0/270183047 learned_addr learned my addr 172.21.15.50:0/270183047 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.686+0000 7f312cc00640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be01e0 0x55f630be83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3127e00640 1 -- 172.21.15.50:0/270183047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be01e0 msgr2=0x55f630be83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3127e00640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be01e0 0x55f630be83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3127e00640 1 -- 172.21.15.50:0/270183047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 msgr2=0x55f630be8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3127e00640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630be8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3127e00640 1 -- 172.21.15.50:0/270183047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f630925620 con 0x55f630be0b00 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f312d600640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630be8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3127e00640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 0x55f630be88e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3114005fd0 tx=0x7f31140cc800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a382d591b71de578 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f310e000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f310e000640 1 -- 172.21.15.50:0/270183047 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140b3070 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31140a42a0 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.687+0000 7f310e000640 1 -- 172.21.15.50:0/270183047 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f3114097ca0 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f310e000640 1 -- 172.21.15.50:0/270183047 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3114020070 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 10 client.25664 fetch_fsmap learned FSMap version 31 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f630bf9bc0 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f310e000640 1 -- 172.21.15.50:0/270183047 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31140a42a0 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f310e000640 1 -- 172.21.15.50:0/270183047 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f31140afc70 con 0x55f630be0b00 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 10 client.25664 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 10 client.25664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 20 client.25664 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630bfaf10 0x55f630bfd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 4 client.25664 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.688+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f630b53830 con 0x55f630bfaf10 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.689+0000 7f312cc00640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630bfaf10 0x55f630bfd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.689+0000 7f310e000640 10 client.25664 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.689+0000 7f312cc00640 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630bfaf10 0x55f630bfd300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e609d8ae08bd6a65 server_cookie=5f0a1fa9a7228d3a in_seq=0 out_seq=0 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.690+0000 7f310e000640 1 -- 172.21.15.50:0/270183047 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f630b53830 con 0x55f630bfaf10 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.690+0000 7f310e000640 10 client.25664 handle_command_reply: tid=0 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.690+0000 7f3132666740 1 client.25664 shutdown 2024-09-12T03:32:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.690+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 msgr2=0x55f630be88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.690+0000 7f3132666740 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 0x55f630be88e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3114005fd0 tx=0x7f31140cc800 comp rx=0 tx=0).stop 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 shutdown_connections 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630be8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be0b00 0x55f630be88e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630bfaf10 0x55f630bfd300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 --2- 172.21.15.50:0/270183047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be01e0 0x55f630be83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f310e000640 0 client.25664 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 >> 172.21.15.50:0/270183047 conn(0x55f630bdc750 msgr2=0x55f630be24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 shutdown_connections 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 1 -- 172.21.15.50:0/270183047 wait complete. 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.691+0000 7f3132666740 20 client.25664 trim_cache size 0 max 16384 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.720+0000 7f3132666740 1 Processor -- start 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.720+0000 7f3132666740 1 -- start start 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3132666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630bf69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3132666740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3132666740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630c00480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3132666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31281b7f30 con 0x55f630be01e0 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3132666740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31281b57d0 con 0x55f630beb3a0 2024-09-12T03:32:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3132666740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31281b3680 con 0x55f630be0b00 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3127e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f312cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630bf69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3127e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53070/0 (socket says 172.21.15.50:53070) 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f3127e00640 1 -- 172.21.15.50:0/3129316654 learned_addr learned my addr 172.21.15.50:0/3129316654 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f312cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630bf69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43714/0 (socket says 172.21.15.50:43714) 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.721+0000 7f312d600640 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630c00480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3127e00640 1 -- 172.21.15.50:0/3129316654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 msgr2=0x55f630c00480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3127e00640 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630c00480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3127e00640 1 -- 172.21.15.50:0/3129316654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 msgr2=0x55f630bf69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3127e00640 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630bf69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3127e00640 1 -- 172.21.15.50:0/3129316654 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f630beae50 con 0x55f630be0b00 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f312d600640 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630c00480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3127e00640 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfff40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3114009610 tx=0x7f3114097a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf09868bb8c9859 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3119200640 1 -- 172.21.15.50:0/3129316654 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140b3070 con 0x55f630be0b00 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3119200640 1 -- 172.21.15.50:0/3129316654 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3114097ca0 con 0x55f630be0b00 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.722+0000 7f3119200640 1 -- 172.21.15.50:0/3129316654 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114084dd0 con 0x55f630be0b00 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 -- 172.21.15.50:0/3129316654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 msgr2=0x55f630bfff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfff40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3114009610 tx=0x7f3114097a00 comp rx=0 tx=0).stop 2024-09-12T03:32:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 -- 172.21.15.50:0/3129316654 shutdown_connections 2024-09-12T03:32:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630beb3a0 0x55f630c00480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfff40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 --2- 172.21.15.50:0/3129316654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630be01e0 0x55f630bf69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 -- 172.21.15.50:0/3129316654 >> 172.21.15.50:0/3129316654 conn(0x55f630bdc750 msgr2=0x55f630bf62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 -- 172.21.15.50:0/3129316654 shutdown_connections 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.723+0000 7f3132666740 1 -- 172.21.15.50:0/3129316654 wait complete. 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.724+0000 7f3132666740 1 -- start start 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.724+0000 7f3132666740 1 Processor -- start 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be01e0 0x55f630bfd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630c63840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31281b3cd0 con 0x55f630beb3a0 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31281b5100 con 0x55f630be01e0 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31281b4360 con 0x55f630be0b00 2024-09-12T03:32:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3132666740 10 client.0 _ll_register_callbacks cb 0x7f3120009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3127e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be01e0 0x55f630bfd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3127e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:53086/0 (socket says 172.21.15.50:53086) 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3127e00640 1 -- 172.21.15.50:0/2758002027 learned_addr learned my addr 172.21.15.50:0/2758002027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312d600640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630c63840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 -- 172.21.15.50:0/2758002027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 msgr2=0x55f630bfd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 -- 172.21.15.50:0/2758002027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 msgr2=0x55f630c63840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630c63840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 -- 172.21.15.50:0/2758002027 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f630c63d80 con 0x55f630be01e0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f3127e00640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f312cc00640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be01e0 0x55f630bfd060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f311c009860 tx=0x7f311c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a89f71445840cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.725+0000 7f310e000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f310e000640 1 -- 172.21.15.50:0/2758002027 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c043040 con 0x55f630be01e0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f310e000640 1 -- 172.21.15.50:0/2758002027 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f311c005230 con 0x55f630be01e0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3114097ca0 con 0x55f630be01e0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f310e000640 1 -- 172.21.15.50:0/2758002027 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f311c037040 con 0x55f630be01e0 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 10 client.25670 fetch_fsmap learned FSMap version 31 2024-09-12T03:32:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f630c64ee0 con 0x55f630be01e0 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f310e000640 1 -- 172.21.15.50:0/2758002027 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3114097ca0 con 0x55f630be01e0 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f310e000640 1 -- 172.21.15.50:0/2758002027 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f311c049040 con 0x55f630be01e0 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 10 client.25670 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 20 client.25670 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630c65660 0x55f630c67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 4 client.25670 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.726+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f630b4db80 con 0x55f630c65660 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.727+0000 7f3127e00640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630c65660 0x55f630c67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.727+0000 7f3127e00640 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630c65660 0x55f630c67a80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d7ccb9ca27d5f7 server_cookie=8a0615d102f0662 in_seq=0 out_seq=0 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.727+0000 7f310e000640 10 client.25670 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f310e000640 1 -- 172.21.15.50:0/2758002027 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f630b4db80 con 0x55f630c65660 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f310e000640 10 client.25670 handle_command_reply: tid=0 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 client.25670 shutdown 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be01e0 msgr2=0x55f630bfd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be01e0 0x55f630bfd060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f311c009860 tx=0x7f311c005f90 comp rx=0 tx=0).stop 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 shutdown_connections 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f630beb3a0 0x55f630c63840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55f630c65660 0x55f630c67a80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f630be0b00 0x55f630bfd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 --2- 172.21.15.50:0/2758002027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f630be01e0 0x55f630bfd060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f310e000640 0 client.25670 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 >> 172.21.15.50:0/2758002027 conn(0x55f630bdc750 msgr2=0x55f630bdb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.744+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 shutdown_connections 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f3132666740 1 -- 172.21.15.50:0/2758002027 wait complete. 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f3132666740 20 client.25670 trim_cache size 0 max 16384 2024-09-12T03:32:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:32:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:32:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:32:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f31100a2e20 msgr2=0x7f31100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f31100a2e20 0x7f31100a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f311c000c00 tx=0x7f311c005d60 comp rx=0 tx=0).stop 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 msgr2=0x7f312819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312819aaa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f311400d950 tx=0x7f311400de10 comp rx=0 tx=0).stop 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 shutdown_connections 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128107b70 0x7f312819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3128102470 0x7f312819aaa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f31100a2e20 0x7f31100a52e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 --2- 172.21.15.50:0/3166060780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3128101ad0 0x7f312819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.745+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 >> 172.21.15.50:0/3166060780 conn(0x7f31280fd7d0 msgr2=0x7f31280fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.746+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 shutdown_connections 2024-09-12T03:32:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:35.746+0000 7f312f600640 1 -- 172.21.15.50:0/3166060780 wait complete. 2024-09-12T03:32:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:37 smithi142 ceph-mon[35329]: pgmap v166: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 372 op/s 2024-09-12T03:32:37.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:37 smithi082 ceph-mon[35046]: pgmap v166: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 372 op/s 2024-09-12T03:32:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:37 smithi050 ceph-mon[30378]: pgmap v166: 129 pgs: 129 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 372 op/s 2024-09-12T03:32:39.015 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x63ed6b9 2024-09-12T03:32:39.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:39 smithi142 ceph-mon[35329]: pgmap v167: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 546 op/s 2024-09-12T03:32:39.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:39 smithi082 ceph-mon[35046]: pgmap v167: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 546 op/s 2024-09-12T03:32:39.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:39 smithi050 ceph-mon[30378]: pgmap v167: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 546 op/s 2024-09-12T03:32:40.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:40 smithi142 ceph-mon[35329]: pgmap v168: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 386 op/s 2024-09-12T03:32:40.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:40 smithi082 ceph-mon[35046]: pgmap v168: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 386 op/s 2024-09-12T03:32:40.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:40 smithi050 ceph-mon[30378]: pgmap v168: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 386 op/s 2024-09-12T03:32:42.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:42 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:32:42.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:42 smithi142 ceph-mon[35329]: pgmap v169: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 370 op/s 2024-09-12T03:32:42.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:42 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:32:42.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:42 smithi082 ceph-mon[35046]: pgmap v169: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 370 op/s 2024-09-12T03:32:42.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:42 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:32:42.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:42 smithi050 ceph-mon[30378]: pgmap v169: 129 pgs: 129 active+clean; 95 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 370 op/s 2024-09-12T03:32:44.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:44 smithi082 ceph-mon[35046]: pgmap v170: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 548 op/s 2024-09-12T03:32:44.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:44 smithi050 ceph-mon[30378]: pgmap v170: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 548 op/s 2024-09-12T03:32:44.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:44 smithi142 ceph-mon[35329]: pgmap v170: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 548 op/s 2024-09-12T03:32:46.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:46 smithi082 ceph-mon[35046]: pgmap v171: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-09-12T03:32:46.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:46 smithi050 ceph-mon[30378]: pgmap v171: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-09-12T03:32:46.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:46 smithi142 ceph-mon[35329]: pgmap v171: 129 pgs: 129 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-09-12T03:32:48.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:48 smithi082 ceph-mon[35046]: pgmap v172: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.7 MiB/s wr, 530 op/s 2024-09-12T03:32:48.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:48 smithi050 ceph-mon[30378]: pgmap v172: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.7 MiB/s wr, 530 op/s 2024-09-12T03:32:48.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:48 smithi142 ceph-mon[35329]: pgmap v172: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.7 MiB/s wr, 530 op/s 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.916+0000 7f5a30400640 1 -- 172.21.15.50:0/2247490160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a280ffdf0 msgr2=0x7f5a281001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.916+0000 7f5a30400640 1 --2- 172.21.15.50:0/2247490160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a280ffdf0 0x7f5a281001d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009900 tx=0x7f5a1c02a480 comp rx=0 tx=0).stop 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 -- 172.21.15.50:0/2247490160 shutdown_connections 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 --2- 172.21.15.50:0/2247490160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 0x7f5a28110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 --2- 172.21.15.50:0/2247490160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a28100710 0x7f5a28109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 --2- 172.21.15.50:0/2247490160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a280ffdf0 0x7f5a281001d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 -- 172.21.15.50:0/2247490160 >> 172.21.15.50:0/2247490160 conn(0x7f5a280fba30 msgr2=0x7f5a280fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 -- 172.21.15.50:0/2247490160 shutdown_connections 2024-09-12T03:32:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.917+0000 7f5a30400640 1 -- 172.21.15.50:0/2247490160 wait complete. 2024-09-12T03:32:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 Processor -- start 2024-09-12T03:32:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 -- start start 2024-09-12T03:32:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a280ffdf0 0x7f5a2819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 0x7f5a2819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 0x7f5a2819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28114110 con 0x7f5a2810a2c0 2024-09-12T03:32:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28113f90 con 0x7f5a28100710 2024-09-12T03:32:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.918+0000 7f5a30400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28114290 con 0x7f5a280ffdf0 2024-09-12T03:32:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.919+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 0x7f5a2819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.919+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 0x7f5a2819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34712/0 (socket says 172.21.15.50:34712) 2024-09-12T03:32:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.919+0000 7f5a2d000640 1 -- 172.21.15.50:0/4133612960 learned_addr learned my addr 172.21.15.50:0/4133612960 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.919+0000 7f5a2e400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 0x7f5a2819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2d000640 1 -- 172.21.15.50:0/4133612960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a280ffdf0 msgr2=0x7f5a2819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2d000640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a280ffdf0 0x7f5a2819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2d000640 1 -- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 msgr2=0x7f5a2819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2d000640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 0x7f5a2819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2d000640 1 -- 172.21.15.50:0/4133612960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2819f580 con 0x7f5a28100710 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2e400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 0x7f5a2819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.920+0000 7f5a2d000640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 0x7f5a2819aad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400e9f0 tx=0x7f5a1400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.921+0000 7f5a26000640 1 -- 172.21.15.50:0/4133612960 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400cd80 con 0x7f5a28100710 2024-09-12T03:32:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.921+0000 7f5a26000640 1 -- 172.21.15.50:0/4133612960 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a14005500 con 0x7f5a28100710 2024-09-12T03:32:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.921+0000 7f5a26000640 1 -- 172.21.15.50:0/4133612960 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14010690 con 0x7f5a28100710 2024-09-12T03:32:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.921+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2819f870 con 0x7f5a28100710 2024-09-12T03:32:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.921+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281a7100 con 0x7f5a28100710 2024-09-12T03:32:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.925+0000 7f5a26000640 1 -- 172.21.15.50:0/4133612960 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5a140046c0 con 0x7f5a28100710 2024-09-12T03:32:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.927+0000 7f5a3344d740 1 Processor -- start 2024-09-12T03:32:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.927+0000 7f5a3344d740 1 -- start start 2024-09-12T03:32:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbb1e0 0x565119dbb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119dd0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a3344d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x565119d21100 con 0x565119dbbb00 2024-09-12T03:32:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a3344d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565119cc91b0 con 0x565119dbb1e0 2024-09-12T03:32:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a3344d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565119d1d0c0 con 0x565119dcebb0 2024-09-12T03:32:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a26000640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f5a100a2f80 0x7f5a100a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119dd0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119dd0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:40500/0 (socket says 172.21.15.50:40500) 2024-09-12T03:32:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a2d000640 1 -- 172.21.15.50:0/1367307235 learned_addr learned my addr 172.21.15.50:0/1367307235 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.928+0000 7f5a2e400640 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbb1e0 0x565119dbb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f5a100a2f80 0x7f5a100a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 -- 172.21.15.50:0/1367307235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 msgr2=0x565119dd0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a26000640 1 -- 172.21.15.50:0/4133612960 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f5a140cfc80 con 0x7f5a28100710 2024-09-12T03:32:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119dd0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 -- 172.21.15.50:0/1367307235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 msgr2=0x565119dc5e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:32:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc5e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 -- 172.21.15.50:0/1367307235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565119a34210 con 0x565119dbb1e0 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.929+0000 7f5a2e400640 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbb1e0 0x565119dbb5c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a20012e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a17b58cece95b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.930+0000 7f5a1aa00640 1 -- 172.21.15.50:0/1367307235 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001eb60 con 0x565119dbb1e0 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.930+0000 7f5a2e400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f5a100a2f80 0x7f5a100a5440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a20022000 tx=0x7f5a2001e000 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.930+0000 7f5a1aa00640 1 -- 172.21.15.50:0/1367307235 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a20023070 con 0x565119dbb1e0 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.930+0000 7f5a1aa00640 1 -- 172.21.15.50:0/1367307235 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000ee30 con 0x565119dbb1e0 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 -- 172.21.15.50:0/1367307235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbb1e0 msgr2=0x565119dbb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbb1e0 0x565119dbb5c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a20012e80 comp rx=0 tx=0).stop 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 -- 172.21.15.50:0/1367307235 shutdown_connections 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119dd0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1367307235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbb1e0 0x565119dbb5c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 -- 172.21.15.50:0/1367307235 >> 172.21.15.50:0/1367307235 conn(0x565119db7750 msgr2=0x565119db7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 -- 172.21.15.50:0/1367307235 shutdown_connections 2024-09-12T03:32:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.931+0000 7f5a3344d740 1 -- 172.21.15.50:0/1367307235 wait complete. 2024-09-12T03:32:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.932+0000 7f5a3344d740 1 -- start start 2024-09-12T03:32:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.932+0000 7f5a3344d740 1 Processor -- start 2024-09-12T03:32:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dcebb0 0x565119dc3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x565119dd17d0 con 0x565119dbbb00 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565119caa090 con 0x565119dcebb0 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565119dd1950 con 0x565119dbb1e0 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a3344d740 10 client.0 _ll_register_callbacks cb 0x565119cfef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2e400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dcebb0 0x565119dc3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dc32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:40510/0 (socket says 172.21.15.50:40510) 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 -- 172.21.15.50:0/1090267103 learned_addr learned my addr 172.21.15.50:0/1090267103 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2d000640 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 -- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dcebb0 msgr2=0x565119dc3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dcebb0 0x565119dc3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 -- 172.21.15.50:0/1090267103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 msgr2=0x565119dc3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.934+0000 7f5a2da00640 1 -- 172.21.15.50:0/1090267103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565119b00620 con 0x565119dbb1e0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a2da00640 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dc32d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c003820 tx=0x7f5a1c003850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb94dcb765fd87bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a0f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a0f400640 1 -- 172.21.15.50:0/1090267103 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c03e070 con 0x565119dbb1e0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a0f400640 1 -- 172.21.15.50:0/1090267103 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f5a1c003e00 con 0x565119dbb1e0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a20023070 con 0x565119dbb1e0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a0f400640 1 -- 172.21.15.50:0/1090267103 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c04a050 con 0x565119dbb1e0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a3344d740 10 client.25666 fetch_fsmap learned FSMap version 32 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565119ddd470 con 0x565119dbb1e0 2024-09-12T03:32:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.935+0000 7f5a0f400640 1 -- 172.21.15.50:0/1090267103 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a20023070 con 0x565119dbb1e0 2024-09-12T03:32:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a0f400640 1 -- 172.21.15.50:0/1090267103 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5a1c004960 con 0x565119dbb1e0 2024-09-12T03:32:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a3344d740 10 client.25666 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a3344d740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a3344d740 20 client.25666 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119de0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a3344d740 4 client.25666 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565119d2e830 con 0x565119dde8e0 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.936+0000 7f5a2d000640 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119de0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.937+0000 7f5a0f400640 10 client.25666 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.937+0000 7f5a2d000640 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119de0d30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d81b84628cd20dd server_cookie=c1b88432eaff654b in_seq=0 out_seq=0 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.937+0000 7f5a0f400640 1 -- 172.21.15.50:0/1090267103 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565119d2e830 con 0x565119dde8e0 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.937+0000 7f5a0f400640 10 client.25666 handle_command_reply: tid=0 2024-09-12T03:32:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 client.25666 shutdown 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 msgr2=0x565119dc32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dc32d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c003820 tx=0x7f5a1c003850 comp rx=0 tx=0).stop 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 shutdown_connections 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dcebb0 0x565119dc3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbbb00 0x565119dc3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119de0d30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 --2- 172.21.15.50:0/1090267103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dc32d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a0f400640 0 client.25666 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.938+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 >> 172.21.15.50:0/1090267103 conn(0x565119db7750 msgr2=0x565119dbd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.939+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 shutdown_connections 2024-09-12T03:32:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.939+0000 7f5a3344d740 1 -- 172.21.15.50:0/1090267103 wait complete. 2024-09-12T03:32:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.939+0000 7f5a3344d740 20 client.25666 trim_cache size 0 max 16384 2024-09-12T03:32:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 Processor -- start 2024-09-12T03:32:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 -- start start 2024-09-12T03:32:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbb1e0 0x565119dc5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dbdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119db28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281af620 con 0x565119dbb1e0 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281acef0 con 0x565119dbbb00 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.963+0000 7f5a3344d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281aad80 con 0x565119dcebb0 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dbdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbb1e0 0x565119dc5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dbdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34742/0 (socket says 172.21.15.50:34742) 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 -- 172.21.15.50:0/2902803530 learned_addr learned my addr 172.21.15.50:0/2902803530 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2e400640 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119db28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 -- 172.21.15.50:0/2902803530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 msgr2=0x565119db28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119db28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 -- 172.21.15.50:0/2902803530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbb1e0 msgr2=0x565119dc5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbb1e0 0x565119dc5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 -- 172.21.15.50:0/2902803530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565119dbb010 con 0x565119dbbb00 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.964+0000 7f5a2d000640 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dbdca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5a14098780 tx=0x7f5a14004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f8fdadabcd30d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.965+0000 7f5a1a000640 1 -- 172.21.15.50:0/2902803530 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1409a110 con 0x565119dbbb00 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.965+0000 7f5a1a000640 1 -- 172.21.15.50:0/2902803530 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1409f070 con 0x565119dbbb00 2024-09-12T03:32:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.965+0000 7f5a1a000640 1 -- 172.21.15.50:0/2902803530 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1409a840 con 0x565119dbbb00 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 -- 172.21.15.50:0/2902803530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 msgr2=0x565119dbdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dbdca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5a14098780 tx=0x7f5a14004b60 comp rx=0 tx=0).stop 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 -- 172.21.15.50:0/2902803530 shutdown_connections 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dcebb0 0x565119db28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dbdca0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 --2- 172.21.15.50:0/2902803530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dbb1e0 0x565119dc5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 -- 172.21.15.50:0/2902803530 >> 172.21.15.50:0/2902803530 conn(0x565119db7750 msgr2=0x565119db6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 -- 172.21.15.50:0/2902803530 shutdown_connections 2024-09-12T03:32:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 -- 172.21.15.50:0/2902803530 wait complete. 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.966+0000 7f5a3344d740 1 -- start start 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 Processor -- start 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dd6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dd73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dcebb0 0x565119dbfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281ab3d0 con 0x565119dcebb0 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281ac820 con 0x565119dbbb00 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281aba80 con 0x565119dbb1e0 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a3344d740 10 client.0 _ll_register_callbacks cb 0x7f5a200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dd6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.967+0000 7f5a2e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dcebb0 0x565119dbfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dd6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:40530/0 (socket says 172.21.15.50:40530) 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2da00640 1 -- 172.21.15.50:0/473412705 learned_addr learned my addr 172.21.15.50:0/473412705 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2d000640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dd73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2e400640 1 -- 172.21.15.50:0/473412705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 msgr2=0x565119dd6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2e400640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dd6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2e400640 1 -- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 msgr2=0x565119dd73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2e400640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dd73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2e400640 1 -- 172.21.15.50:0/473412705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565119dc0540 con 0x565119dcebb0 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2d000640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dd73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a2e400640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dcebb0 0x565119dbfcc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009a00 tx=0x7f5a200128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f438a7639611be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a0f400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.968+0000 7f5a0f400640 1 -- 172.21.15.50:0/473412705 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001eb60 con 0x565119dcebb0 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a0f400640 1 -- 172.21.15.50:0/473412705 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f5a2001ed00 con 0x565119dcebb0 2024-09-12T03:32:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1409f070 con 0x565119dcebb0 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a0f400640 1 -- 172.21.15.50:0/473412705 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a20029460 con 0x565119dcebb0 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 10 client.15648 fetch_fsmap learned FSMap version 32 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565119e49230 con 0x565119dcebb0 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a0f400640 1 -- 172.21.15.50:0/473412705 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1409f070 con 0x565119dcebb0 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a0f400640 1 -- 172.21.15.50:0/473412705 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5a2003b030 con 0x565119dcebb0 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 10 client.15648 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:32:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 20 client.15648 populate_metadata read hostname 'smithi050' 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119e4b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 4 client.15648 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.969+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565119d28b80 con 0x565119dde8e0 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.970+0000 7f5a2da00640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119e4b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.970+0000 7f5a2da00640 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119e4b8d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92cae34041255e0 server_cookie=144a3b9ad9f5e245 in_seq=0 out_seq=0 2024-09-12T03:32:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.970+0000 7f5a0f400640 10 client.15648 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.989+0000 7f5a0f400640 1 -- 172.21.15.50:0/473412705 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565119d28b80 con 0x565119dde8e0 2024-09-12T03:32:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.989+0000 7f5a0f400640 10 client.15648 handle_command_reply: tid=0 2024-09-12T03:32:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.990+0000 7f5a3344d740 1 client.15648 shutdown 2024-09-12T03:32:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.990+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dcebb0 msgr2=0x565119dbfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.990+0000 7f5a3344d740 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dcebb0 0x565119dbfcc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009a00 tx=0x7f5a200128a0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 shutdown_connections 2024-09-12T03:32:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565119dcebb0 0x565119dbfcc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565119dbbb00 0x565119dd73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x565119dde8e0 0x565119e4b8d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 --2- 172.21.15.50:0/473412705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565119dbb1e0 0x565119dd6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 >> 172.21.15.50:0/473412705 conn(0x565119db7750 msgr2=0x565119db6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a0f400640 0 client.15648 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:32:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.992+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 shutdown_connections 2024-09-12T03:32:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.993+0000 7f5a3344d740 1 -- 172.21.15.50:0/473412705 wait complete. 2024-09-12T03:32:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.993+0000 7f5a3344d740 20 client.15648 trim_cache size 0 max 16384 2024-09-12T03:32:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:32:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:32:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:32:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.994+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f5a100a2f80 msgr2=0x7f5a100a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.994+0000 7f5a30400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f5a100a2f80 0x7f5a100a5440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a20022000 tx=0x7f5a2001e000 comp rx=0 tx=0).stop 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.994+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 msgr2=0x7f5a2819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.994+0000 7f5a30400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 0x7f5a2819aad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400e9f0 tx=0x7f5a1400edf0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 shutdown_connections 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f5a100a2f80 0x7f5a100a5440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a2810a2c0 0x7f5a2819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a28100710 0x7f5a2819aad0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 --2- 172.21.15.50:0/4133612960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a280ffdf0 0x7f5a2819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 >> 172.21.15.50:0/4133612960 conn(0x7f5a280fba30 msgr2=0x7f5a280fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 shutdown_connections 2024-09-12T03:32:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:32:48.995+0000 7f5a30400640 1 -- 172.21.15.50:0/4133612960 wait complete. 2024-09-12T03:32:50.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:50 smithi082 ceph-mon[35046]: pgmap v173: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 372 op/s 2024-09-12T03:32:50.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:50 smithi050 ceph-mon[30378]: pgmap v173: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 372 op/s 2024-09-12T03:32:50.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:50 smithi142 ceph-mon[35329]: pgmap v173: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 372 op/s 2024-09-12T03:32:52.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:52 smithi082 ceph-mon[35046]: pgmap v174: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 356 op/s 2024-09-12T03:32:52.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:52 smithi050 ceph-mon[30378]: pgmap v174: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 356 op/s 2024-09-12T03:32:52.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:52 smithi142 ceph-mon[35329]: pgmap v174: 129 pgs: 129 active+clean; 97 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 356 op/s 2024-09-12T03:32:54.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:54 smithi082 ceph-mon[35046]: pgmap v175: 129 pgs: 129 active+clean; 92 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.7 MiB/s wr, 556 op/s 2024-09-12T03:32:54.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:54 smithi050 ceph-mon[30378]: pgmap v175: 129 pgs: 129 active+clean; 92 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.7 MiB/s wr, 556 op/s 2024-09-12T03:32:54.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:54 smithi142 ceph-mon[35329]: pgmap v175: 129 pgs: 129 active+clean; 92 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.7 MiB/s wr, 556 op/s 2024-09-12T03:32:56.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:56 smithi082 ceph-mon[35046]: pgmap v176: 129 pgs: 129 active+clean; 92 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-09-12T03:32:56.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:56 smithi050 ceph-mon[30378]: pgmap v176: 129 pgs: 129 active+clean; 92 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-09-12T03:32:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:56 smithi142 ceph-mon[35329]: pgmap v176: 129 pgs: 129 active+clean; 92 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-09-12T03:32:58.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:32:58 smithi082 ceph-mon[35046]: pgmap v177: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.6 MiB/s wr, 551 op/s 2024-09-12T03:32:58.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:32:58 smithi050 ceph-mon[30378]: pgmap v177: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.6 MiB/s wr, 551 op/s 2024-09-12T03:32:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:32:58 smithi142 ceph-mon[35329]: pgmap v177: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.6 MiB/s wr, 551 op/s 2024-09-12T03:33:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:00 smithi082 ceph-mon[35046]: pgmap v178: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-09-12T03:33:00.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:00 smithi050 ceph-mon[30378]: pgmap v178: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-09-12T03:33:00.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:00 smithi142 ceph-mon[35329]: pgmap v178: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-09-12T03:33:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 -- 172.21.15.50:0/3717431055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d4101370 msgr2=0x7f14d410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 --2- 172.21.15.50:0/3717431055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d4101370 0x7f14d410d8a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ca0 tx=0x7f14cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 -- 172.21.15.50:0/3717431055 shutdown_connections 2024-09-12T03:33:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 --2- 172.21.15.50:0/3717431055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d4101370 0x7f14d410d8a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 --2- 172.21.15.50:0/3717431055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d41009d0 0x7f14d4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 --2- 172.21.15.50:0/3717431055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d41069e0 0x7f14d4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 -- 172.21.15.50:0/3717431055 >> 172.21.15.50:0/3717431055 conn(0x7f14d40fc6b0 msgr2=0x7f14d40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 -- 172.21.15.50:0/3717431055 shutdown_connections 2024-09-12T03:33:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.150+0000 7f14daa00640 1 -- 172.21.15.50:0/3717431055 wait complete. 2024-09-12T03:33:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.151+0000 7f14daa00640 1 Processor -- start 2024-09-12T03:33:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.151+0000 7f14daa00640 1 -- start start 2024-09-12T03:33:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14daa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d41009d0 0x7f14d419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14daa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 0x7f14d419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d41069e0 0x7f14d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14daa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4110e70 con 0x7f14d41069e0 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14daa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4110cf0 con 0x7f14d41009d0 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14daa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4110ff0 con 0x7f14d4101370 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 0x7f14d419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d41009d0 0x7f14d419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d41069e0 0x7f14d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 0x7f14d419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41360/0 (socket says 172.21.15.50:41360) 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 -- 172.21.15.50:0/2064616835 learned_addr learned my addr 172.21.15.50:0/2064616835 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 -- 172.21.15.50:0/2064616835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d41009d0 msgr2=0x7f14d419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d41009d0 0x7f14d419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 -- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d41069e0 msgr2=0x7f14d419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d41069e0 0x7f14d419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.152+0000 7f14d3400640 1 -- 172.21.15.50:0/2064616835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d419f4a0 con 0x7f14d4101370 2024-09-12T03:33:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.153+0000 7f14d3400640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 0x7f14d419a9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e9f0 tx=0x7f14c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.153+0000 7f14d0c00640 1 -- 172.21.15.50:0/2064616835 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c000cd80 con 0x7f14d4101370 2024-09-12T03:33:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.153+0000 7f14d0c00640 1 -- 172.21.15.50:0/2064616835 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c0010910 con 0x7f14d4101370 2024-09-12T03:33:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.153+0000 7f14d0c00640 1 -- 172.21.15.50:0/2064616835 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0018990 con 0x7f14d4101370 2024-09-12T03:33:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.153+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d419f790 con 0x7f14d4101370 2024-09-12T03:33:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.153+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41a7020 con 0x7f14d4101370 2024-09-12T03:33:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.164+0000 7f14d0c00640 1 -- 172.21.15.50:0/2064616835 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f14c0010ac0 con 0x7f14d4101370 2024-09-12T03:33:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.167+0000 7f14dd9ce740 1 Processor -- start 2024-09-12T03:33:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.168+0000 7f14d0c00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f14b80a2f00 0x7f14b80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.169+0000 7f14dd9ce740 1 -- start start 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14d8a00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f14b80a2f00 0x7f14b80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945c05dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c10670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c21bc0 0x555945c23fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555945b63100 con 0x555945c21bc0 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555945b0b1b0 con 0x555945c06310 2024-09-12T03:33:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.170+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555945b5f0c0 con 0x555945c059f0 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.171+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945c05dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.171+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945c05dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41362/0 (socket says 172.21.15.50:41362) 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.171+0000 7f14d3e00640 1 -- 172.21.15.50:0/519232333 learned_addr learned my addr 172.21.15.50:0/519232333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.171+0000 7f14d8a00640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c21bc0 0x555945c23fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.172+0000 7f14d0c00640 1 -- 172.21.15.50:0/2064616835 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f14c00cf9c0 con 0x7f14d4101370 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.172+0000 7f14d3e00640 1 -- 172.21.15.50:0/519232333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 msgr2=0x555945c10670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.172+0000 7f14d8a00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f14b80a2f00 0x7f14b80a53c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006010 tx=0x7f14cc005d80 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d3400640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c10670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d3e00640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c10670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d3e00640 1 -- 172.21.15.50:0/519232333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c21bc0 msgr2=0x555945c23fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d3e00640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c21bc0 0x555945c23fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d3e00640 1 -- 172.21.15.50:0/519232333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555945876210 con 0x555945c059f0 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d8a00640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c21bc0 0x555945c23fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.173+0000 7f14d3400640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c10670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.174+0000 7f14d3e00640 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945c05dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009c70 tx=0x7f14c4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=283d3ad9f6d13d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.174+0000 7f14c9200640 1 -- 172.21.15.50:0/519232333 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4004230 con 0x555945c059f0 2024-09-12T03:33:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.174+0000 7f14c9200640 1 -- 172.21.15.50:0/519232333 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c40043d0 con 0x555945c059f0 2024-09-12T03:33:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.174+0000 7f14c9200640 1 -- 172.21.15.50:0/519232333 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4019790 con 0x555945c059f0 2024-09-12T03:33:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.175+0000 7f14dd9ce740 1 -- 172.21.15.50:0/519232333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 msgr2=0x555945c05dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.175+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945c05dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009c70 tx=0x7f14c4011810 comp rx=0 tx=0).stop 2024-09-12T03:33:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.176+0000 7f14dd9ce740 1 -- 172.21.15.50:0/519232333 shutdown_connections 2024-09-12T03:33:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.176+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c21bc0 0x555945c23fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.176+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c10670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.176+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/519232333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945c05dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.176+0000 7f14dd9ce740 1 -- 172.21.15.50:0/519232333 >> 172.21.15.50:0/519232333 conn(0x555945c01f60 msgr2=0x555945c02340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.177+0000 7f14dd9ce740 1 -- 172.21.15.50:0/519232333 shutdown_connections 2024-09-12T03:33:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.177+0000 7f14dd9ce740 1 -- 172.21.15.50:0/519232333 wait complete. 2024-09-12T03:33:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.177+0000 7f14dd9ce740 1 -- start start 2024-09-12T03:33:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.178+0000 7f14dd9ce740 1 Processor -- start 2024-09-12T03:33:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c059f0 0x555945c0da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555945c24790 con 0x555945c06310 2024-09-12T03:33:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555945aec090 con 0x555945c059f0 2024-09-12T03:33:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555945c24910 con 0x555945c21bc0 2024-09-12T03:33:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14dd9ce740 10 client.0 _ll_register_callbacks cb 0x555945b40f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c059f0 0x555945c0da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c0e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41374/0 (socket says 172.21.15.50:41374) 2024-09-12T03:33:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.179+0000 7f14d8a00640 1 -- 172.21.15.50:0/1057672580 learned_addr learned my addr 172.21.15.50:0/1057672580 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d8a00640 1 -- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c059f0 msgr2=0x555945c0da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d8a00640 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c059f0 0x555945c0da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d8a00640 1 -- 172.21.15.50:0/1057672580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 msgr2=0x555945c0dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d8a00640 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d8a00640 1 -- 172.21.15.50:0/1057672580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555945942620 con 0x555945c21bc0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d3e00640 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c059f0 0x555945c0da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14d8a00640 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c0e510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ca0 tx=0x7f14cc02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2375b8647d636239 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14bd600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.180+0000 7f14bd600640 1 -- 172.21.15.50:0/1057672580 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc031660 con 0x555945c21bc0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14bd600640 1 -- 172.21.15.50:0/1057672580 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f14cc031800 con 0x555945c21bc0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c40043d0 con 0x555945c21bc0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14bd600640 1 -- 172.21.15.50:0/1057672580 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f14cc004040 con 0x555945c21bc0 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 10 client.25684 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555945c30350 con 0x555945c21bc0 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14bd600640 1 -- 172.21.15.50:0/1057672580 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14cc04f070 con 0x555945c21bc0 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14bd600640 1 -- 172.21.15.50:0/1057672580 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f14cc02fa50 con 0x555945c21bc0 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 10 client.25684 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 20 client.25684 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.181+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c31850 0x555945c33c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.182+0000 7f14dd9ce740 4 client.25684 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.182+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f14c40043d0 con 0x555945c31850 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.182+0000 7f14d3e00640 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c31850 0x555945c33c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.182+0000 7f14d3e00640 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c31850 0x555945c33c80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d066b1a14a986371 server_cookie=8fc212a55da67358 in_seq=0 out_seq=0 2024-09-12T03:33:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.182+0000 7f14bd600640 10 client.25684 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14bd600640 1 -- 172.21.15.50:0/1057672580 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f14c40043d0 con 0x555945c31850 2024-09-12T03:33:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14bd600640 10 client.25684 handle_command_reply: tid=0 2024-09-12T03:33:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 client.25684 shutdown 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 msgr2=0x555945c0e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c0e510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ca0 tx=0x7f14cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 shutdown_connections 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c31850 0x555945c33c80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c0e510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/1057672580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c059f0 0x555945c0da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.183+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 >> 172.21.15.50:0/1057672580 conn(0x555945c01f60 msgr2=0x555945bf8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.184+0000 7f14bd600640 0 client.25684 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.184+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 shutdown_connections 2024-09-12T03:33:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.184+0000 7f14dd9ce740 1 -- 172.21.15.50:0/1057672580 wait complete. 2024-09-12T03:33:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.184+0000 7f14dd9ce740 20 client.25684 trim_cache size 0 max 16384 2024-09-12T03:33:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 Processor -- start 2024-09-12T03:33:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 -- start start 2024-09-12T03:33:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945bf46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c21bc0 0x555945c39590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4074030 con 0x555945c06310 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4071900 con 0x555945c21bc0 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.203+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14d406f790 con 0x555945c059f0 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945bf46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945bf46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41382/0 (socket says 172.21.15.50:41382) 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 -- 172.21.15.50:0/722838965 learned_addr learned my addr 172.21.15.50:0/722838965 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3400640 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d8a00640 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c21bc0 0x555945c39590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 -- 172.21.15.50:0/722838965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c21bc0 msgr2=0x555945c39590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c21bc0 0x555945c39590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 -- 172.21.15.50:0/722838965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 msgr2=0x555945c0e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 -- 172.21.15.50:0/722838965 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555945c2da30 con 0x555945c059f0 2024-09-12T03:33:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.204+0000 7f14d3e00640 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945bf46a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009c70 tx=0x7f14c4004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8cd2cfe4aea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.205+0000 7f14bfe00640 1 -- 172.21.15.50:0/722838965 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4032070 con 0x555945c059f0 2024-09-12T03:33:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.205+0000 7f14bfe00640 1 -- 172.21.15.50:0/722838965 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c402a070 con 0x555945c059f0 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.205+0000 7f14bfe00640 1 -- 172.21.15.50:0/722838965 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c40035f0 con 0x555945c059f0 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.205+0000 7f14dd9ce740 1 -- 172.21.15.50:0/722838965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 msgr2=0x555945bf46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.205+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945bf46a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009c70 tx=0x7f14c4004580 comp rx=0 tx=0).stop 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 -- 172.21.15.50:0/722838965 shutdown_connections 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c21bc0 0x555945c39590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c06310 0x555945c0e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/722838965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c059f0 0x555945bf46a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 -- 172.21.15.50:0/722838965 >> 172.21.15.50:0/722838965 conn(0x555945c01f60 msgr2=0x555945c0c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 -- 172.21.15.50:0/722838965 shutdown_connections 2024-09-12T03:33:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 -- 172.21.15.50:0/722838965 wait complete. 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 -- start start 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.206+0000 7f14dd9ce740 1 Processor -- start 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 0x555945c24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c25240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c9a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14d406fde0 con 0x555945c059f0 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4071230 con 0x555945c06310 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4070490 con 0x555945c21bc0 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14dd9ce740 10 client.0 _ll_register_callbacks cb 0x7f14cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c9a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 0x555945c24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c25240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c9a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41384/0 (socket says 172.21.15.50:41384) 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 -- 172.21.15.50:0/3195232610 learned_addr learned my addr 172.21.15.50:0/3195232610 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 0x555945c24d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37870/0 (socket says 172.21.15.50:37870) 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 -- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 msgr2=0x555945c25240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c25240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 -- 172.21.15.50:0/3195232610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 msgr2=0x555945c24d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 0x555945c24d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d8a00640 1 -- 172.21.15.50:0/3195232610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555945c9aa90 con 0x555945c21bc0 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.207+0000 7f14d3e00640 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 0x555945c24d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14d8a00640 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c9a550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009940 tx=0x7f14cc02f8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccaebfdb16b3e8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14bd600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14bd600640 1 -- 172.21.15.50:0/3195232610 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc031c30 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14bd600640 1 -- 172.21.15.50:0/3195232610 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f14cc0043e0 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c402a070 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14bd600640 1 -- 172.21.15.50:0/3195232610 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f14cc0045e0 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14dd9ce740 10 client.25690 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555945c9bc80 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.208+0000 7f14bd600640 1 -- 172.21.15.50:0/3195232610 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c402a070 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14bd600640 1 -- 172.21.15.50:0/3195232610 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f14cc054470 con 0x555945c21bc0 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14dd9ce740 10 client.25690 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14dd9ce740 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14dd9ce740 20 client.25690 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c9c480 0x555945c9e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14dd9ce740 4 client.25690 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555945bf4d80 con 0x555945c9c480 2024-09-12T03:33:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14d3e00640 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c9c480 0x555945c9e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14d3e00640 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c9c480 0x555945c9e8a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0de6a1cd09afa88 server_cookie=237a34d6ca5db131 in_seq=0 out_seq=0 2024-09-12T03:33:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.209+0000 7f14bd600640 10 client.25690 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:02.221 INFO:tasks.workunit.client.0.smithi050.stdout:copying to largest ever: 0x63ef9ee 2024-09-12T03:33:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.228+0000 7f14bd600640 1 -- 172.21.15.50:0/3195232610 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555945bf4d80 con 0x555945c9c480 2024-09-12T03:33:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.228+0000 7f14bd600640 10 client.25690 handle_command_reply: tid=0 2024-09-12T03:33:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.228+0000 7f14dd9ce740 1 client.25690 shutdown 2024-09-12T03:33:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.229+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 msgr2=0x555945c9a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.229+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c9a550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009940 tx=0x7f14cc02f8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 shutdown_connections 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555945c9c480 0x555945c9e8a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555945c21bc0 0x555945c9a550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555945c06310 0x555945c25240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14dd9ce740 1 --2- 172.21.15.50:0/3195232610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555945c059f0 0x555945c24d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 >> 172.21.15.50:0/3195232610 conn(0x555945c01f60 msgr2=0x555945bf8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.230+0000 7f14bd600640 0 client.25690 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.231+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 shutdown_connections 2024-09-12T03:33:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.233+0000 7f14dd9ce740 1 -- 172.21.15.50:0/3195232610 wait complete. 2024-09-12T03:33:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.233+0000 7f14dd9ce740 20 client.25690 trim_cache size 0 max 16384 2024-09-12T03:33:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:33:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:33:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:33:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:33:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.236+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f14b80a2f00 msgr2=0x7f14b80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.236+0000 7f14daa00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f14b80a2f00 0x7f14b80a53c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006010 tx=0x7f14cc005d80 comp rx=0 tx=0).stop 2024-09-12T03:33:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.236+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 msgr2=0x7f14d419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.236+0000 7f14daa00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 0x7f14d419a9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e9f0 tx=0x7f14c000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 shutdown_connections 2024-09-12T03:33:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f14b80a2f00 0x7f14b80a53c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d41069e0 0x7f14d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d4101370 0x7f14d419a9f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 --2- 172.21.15.50:0/2064616835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d41009d0 0x7f14d419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 >> 172.21.15.50:0/2064616835 conn(0x7f14d40fc6b0 msgr2=0x7f14d40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.237+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 shutdown_connections 2024-09-12T03:33:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:02.238+0000 7f14daa00640 1 -- 172.21.15.50:0/2064616835 wait complete. 2024-09-12T03:33:02.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:02 smithi082 ceph-mon[35046]: pgmap v179: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-09-12T03:33:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:02 smithi050 ceph-mon[30378]: pgmap v179: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-09-12T03:33:02.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:02 smithi142 ceph-mon[35329]: pgmap v179: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 388 op/s 2024-09-12T03:33:04.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:04 smithi082 ceph-mon[35046]: pgmap v180: 129 pgs: 129 active+clean; 105 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 583 op/s 2024-09-12T03:33:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:04 smithi050 ceph-mon[30378]: pgmap v180: 129 pgs: 129 active+clean; 105 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 583 op/s 2024-09-12T03:33:04.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:04 smithi142 ceph-mon[35329]: pgmap v180: 129 pgs: 129 active+clean; 105 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 583 op/s 2024-09-12T03:33:06.603 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:06 smithi082 ceph-mon[35046]: pgmap v181: 129 pgs: 129 active+clean; 105 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.1 MiB/s wr, 382 op/s 2024-09-12T03:33:06.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:06 smithi050 ceph-mon[30378]: pgmap v181: 129 pgs: 129 active+clean; 105 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.1 MiB/s wr, 382 op/s 2024-09-12T03:33:06.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:06 smithi142 ceph-mon[35329]: pgmap v181: 129 pgs: 129 active+clean; 105 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.1 MiB/s wr, 382 op/s 2024-09-12T03:33:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:08 smithi050 ceph-mon[30378]: pgmap v182: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 550 op/s 2024-09-12T03:33:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:08 smithi142 ceph-mon[35329]: pgmap v182: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 550 op/s 2024-09-12T03:33:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:08 smithi082 ceph-mon[35046]: pgmap v182: 129 pgs: 129 active+clean; 122 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 550 op/s 2024-09-12T03:33:10.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:10 smithi050 ceph-mon[30378]: pgmap v183: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 392 op/s 2024-09-12T03:33:10.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:10 smithi142 ceph-mon[35329]: pgmap v183: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 392 op/s 2024-09-12T03:33:10.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:10 smithi082 ceph-mon[35046]: pgmap v183: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 392 op/s 2024-09-12T03:33:12.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:12 smithi050 ceph-mon[30378]: pgmap v184: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2024-09-12T03:33:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:12 smithi142 ceph-mon[35329]: pgmap v184: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2024-09-12T03:33:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:12 smithi082 ceph-mon[35046]: pgmap v184: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2024-09-12T03:33:14.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:14 smithi050 ceph-mon[30378]: pgmap v185: 129 pgs: 129 active+clean; 105 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 556 op/s 2024-09-12T03:33:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:14 smithi142 ceph-mon[35329]: pgmap v185: 129 pgs: 129 active+clean; 105 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 556 op/s 2024-09-12T03:33:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:14 smithi082 ceph-mon[35046]: pgmap v185: 129 pgs: 129 active+clean; 105 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 556 op/s 2024-09-12T03:33:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 -- 172.21.15.50:0/3441068856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102ae0 msgr2=0x7fd848102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 --2- 172.21.15.50:0/3441068856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102ae0 0x7fd848102ec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd838009900 tx=0x7fd83802a480 comp rx=0 tx=0).stop 2024-09-12T03:33:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 -- 172.21.15.50:0/3441068856 shutdown_connections 2024-09-12T03:33:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 --2- 172.21.15.50:0/3441068856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd84810bf00 0x7fd84810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 --2- 172.21.15.50:0/3441068856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd848103400 0x7fd84810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 --2- 172.21.15.50:0/3441068856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102ae0 0x7fd848102ec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 -- 172.21.15.50:0/3441068856 >> 172.21.15.50:0/3441068856 conn(0x7fd8480777c0 msgr2=0x7fd8480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 -- 172.21.15.50:0/3441068856 shutdown_connections 2024-09-12T03:33:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.387+0000 7fd84ca00640 1 -- 172.21.15.50:0/3441068856 wait complete. 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 Processor -- start 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 -- start start 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 0x7fd848100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103400 0x7fd848100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd84810bf00 0x7fd8481011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd848111d20 con 0x7fd84810bf00 2024-09-12T03:33:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd848111ba0 con 0x7fd848103400 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd84ca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd848111ea0 con 0x7fd848102ae0 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd847400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 0x7fd848100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd846a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103400 0x7fd848100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.388+0000 7fd847400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 0x7fd848100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:34208/0 (socket says 172.21.15.50:34208) 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd847400640 1 -- 172.21.15.50:0/2789784333 learned_addr learned my addr 172.21.15.50:0/2789784333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd847e00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd84810bf00 0x7fd8481011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd846a00640 1 -- 172.21.15.50:0/2789784333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 msgr2=0x7fd848100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd846a00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 0x7fd848100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd846a00640 1 -- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd84810bf00 msgr2=0x7fd8481011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd846a00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd84810bf00 0x7fd8481011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd846a00640 1 -- 172.21.15.50:0/2789784333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481a6750 con 0x7fd848103400 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd847400640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 0x7fd848100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.389+0000 7fd846a00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103400 0x7fd848100c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9f0 tx=0x7fd83400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.390+0000 7fd83fe00640 1 -- 172.21.15.50:0/2789784333 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83400ccc0 con 0x7fd848103400 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.390+0000 7fd83fe00640 1 -- 172.21.15.50:0/2789784333 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd834010910 con 0x7fd848103400 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.390+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8481a6a40 con 0x7fd848103400 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.390+0000 7fd83fe00640 1 -- 172.21.15.50:0/2789784333 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834018960 con 0x7fd848103400 2024-09-12T03:33:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.390+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481a6fe0 con 0x7fd848103400 2024-09-12T03:33:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.393+0000 7fd850e21740 1 Processor -- start 2024-09-12T03:33:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.393+0000 7fd850e21740 1 -- start start 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd850e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed62404790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd850e21740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd850e21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed6240f570 0x55ed62411960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd850e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ed6236a100 con 0x55ed624043b0 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd850e21740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ed623121b0 con 0x55ed62404cd0 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd850e21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed623660c0 con 0x55ed6240f570 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd847e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed62404790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd847e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed62404790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48948/0 (socket says 172.21.15.50:48948) 2024-09-12T03:33:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.394+0000 7fd847e00640 1 -- 172.21.15.50:0/3743616382 learned_addr learned my addr 172.21.15.50:0/3743616382 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.395+0000 7fd847e00640 1 -- 172.21.15.50:0/3743616382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed6240f570 msgr2=0x55ed62411960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:33:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.395+0000 7fd847e00640 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed6240f570 0x55ed62411960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.395+0000 7fd847e00640 1 -- 172.21.15.50:0/3743616382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 msgr2=0x55ed6240f030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.395+0000 7fd847e00640 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.395+0000 7fd847e00640 1 -- 172.21.15.50:0/3743616382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed6207d210 con 0x55ed624043b0 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.397+0000 7fd847e00640 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed62404790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd840009f50 tx=0x7fd8400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bdc5d8af1994b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.397+0000 7fd833e00640 1 -- 172.21.15.50:0/3743616382 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840004b20 con 0x55ed624043b0 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.397+0000 7fd83fe00640 1 -- 172.21.15.50:0/2789784333 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd834010ac0 con 0x7fd848103400 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.397+0000 7fd833e00640 1 -- 172.21.15.50:0/3743616382 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd840004cc0 con 0x55ed624043b0 2024-09-12T03:33:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.397+0000 7fd833e00640 1 -- 172.21.15.50:0/3743616382 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840005450 con 0x55ed624043b0 2024-09-12T03:33:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.398+0000 7fd850e21740 1 -- 172.21.15.50:0/3743616382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 msgr2=0x55ed62404790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.398+0000 7fd850e21740 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed62404790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd840009f50 tx=0x7fd8400118d0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 -- 172.21.15.50:0/3743616382 shutdown_connections 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed6240f570 0x55ed62411960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 --2- 172.21.15.50:0/3743616382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed62404790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 -- 172.21.15.50:0/3743616382 >> 172.21.15.50:0/3743616382 conn(0x55ed62400920 msgr2=0x55ed62400d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 -- 172.21.15.50:0/3743616382 shutdown_connections 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 -- 172.21.15.50:0/3743616382 wait complete. 2024-09-12T03:33:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd83fe00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd82c0a2c80 0x7fd82c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.399+0000 7fd850e21740 1 -- start start 2024-09-12T03:33:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.400+0000 7fd83fe00640 1 -- 172.21.15.50:0/2789784333 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fd8340cfb30 con 0x7fd848103400 2024-09-12T03:33:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.400+0000 7fd847400640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd82c0a2c80 0x7fd82c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.400+0000 7fd850e21740 1 Processor -- start 2024-09-12T03:33:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed6240c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6240cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed6240f570 0x55ed6240cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ed62411ff0 con 0x55ed624043b0 2024-09-12T03:33:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ed622f3090 con 0x55ed6240f570 2024-09-12T03:33:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed62412170 con 0x55ed62404cd0 2024-09-12T03:33:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd850e21740 10 client.0 _ll_register_callbacks cb 0x55ed62347f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.401+0000 7fd847400640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd82c0a2c80 0x7fd82c0a5140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8380096c0 tx=0x7fd8380046c0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed6240c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed6240c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48950/0 (socket says 172.21.15.50:48950) 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 -- 172.21.15.50:0/871941570 learned_addr learned my addr 172.21.15.50:0/871941570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847e00640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed6240f570 0x55ed6240cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 -- 172.21.15.50:0/871941570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 msgr2=0x55ed6240cab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6240cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 -- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed6240f570 msgr2=0x55ed6240cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed6240f570 0x55ed6240cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.402+0000 7fd847400640 1 -- 172.21.15.50:0/871941570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed62149620 con 0x55ed624043b0 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.403+0000 7fd847e00640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed6240f570 0x55ed6240cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.403+0000 7fd847400640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed6240c570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd83802aa70 tx=0x7fd838009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13956cf09ff0e4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.403+0000 7fd830c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.403+0000 7fd830c00640 1 -- 172.21.15.50:0/871941570 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83804d020 con 0x55ed624043b0 2024-09-12T03:33:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.404+0000 7fd830c00640 1 -- 172.21.15.50:0/871941570 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fd8380037f0 con 0x55ed624043b0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.404+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd840004cc0 con 0x55ed624043b0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.404+0000 7fd830c00640 1 -- 172.21.15.50:0/871941570 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd838058050 con 0x55ed624043b0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.404+0000 7fd850e21740 10 client.15675 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.404+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ed624266a0 con 0x55ed624043b0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.404+0000 7fd830c00640 1 -- 172.21.15.50:0/871941570 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd840004cc0 con 0x55ed624043b0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd830c00640 1 -- 172.21.15.50:0/871941570 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd8380499b0 con 0x55ed624043b0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd850e21740 10 client.15675 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd850e21740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd850e21740 20 client.15675 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd850e21740 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62427bb0 0x55ed62429fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd850e21740 4 client.15675 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed62377830 con 0x55ed62427bb0 2024-09-12T03:33:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.405+0000 7fd846a00640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62427bb0 0x55ed62429fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd846a00640 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62427bb0 0x55ed62429fc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c21dc97f5adf7a2 server_cookie=dfb15a606a883fa in_seq=0 out_seq=0 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd830c00640 10 client.15675 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd830c00640 1 -- 172.21.15.50:0/871941570 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed62377830 con 0x55ed62427bb0 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd830c00640 10 client.15675 handle_command_reply: tid=0 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd850e21740 1 client.15675 shutdown 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 msgr2=0x55ed6240c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.406+0000 7fd850e21740 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed6240c570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd83802aa70 tx=0x7fd838009260 comp rx=0 tx=0).stop 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 shutdown_connections 2024-09-12T03:33:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed6240f570 0x55ed6240cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62427bb0 0x55ed62429fc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6240cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 --2- 172.21.15.50:0/871941570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed624043b0 0x55ed6240c570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 >> 172.21.15.50:0/871941570 conn(0x55ed62400920 msgr2=0x55ed62406680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd830c00640 0 client.15675 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 shutdown_connections 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 1 -- 172.21.15.50:0/871941570 wait complete. 2024-09-12T03:33:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.407+0000 7fd850e21740 20 client.15675 trim_cache size 0 max 16384 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.423+0000 7fd850e21740 1 Processor -- start 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.423+0000 7fd850e21740 1 -- start start 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd850e21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed624043b0 0x55ed62406e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd850e21740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd850e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6242f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd850e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481af580 con 0x55ed6240f570 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd850e21740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ace50 con 0x55ed62404cd0 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd850e21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481aaca0 con 0x55ed624043b0 2024-09-12T03:33:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59976/0 (socket says 172.21.15.50:59976) 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 -- 172.21.15.50:0/2938176590 learned_addr learned my addr 172.21.15.50:0/2938176590 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd847e00640 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6242f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd847400640 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed624043b0 0x55ed62406e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 -- 172.21.15.50:0/2938176590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed624043b0 msgr2=0x55ed62406e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed624043b0 0x55ed62406e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 -- 172.21.15.50:0/2938176590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 msgr2=0x55ed6242f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6242f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd846a00640 1 -- 172.21.15.50:0/2938176590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed6240f020 con 0x55ed62404cd0 2024-09-12T03:33:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.424+0000 7fd847e00640 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6242f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.425+0000 7fd846a00640 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240e3d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd8340986e0 tx=0x7fd834009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6025892f000e5f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.425+0000 7fd833400640 1 -- 172.21.15.50:0/2938176590 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340a48d0 con 0x55ed62404cd0 2024-09-12T03:33:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.425+0000 7fd833400640 1 -- 172.21.15.50:0/2938176590 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8340b1030 con 0x55ed62404cd0 2024-09-12T03:33:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.425+0000 7fd833400640 1 -- 172.21.15.50:0/2938176590 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83409b3c0 con 0x55ed62404cd0 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 -- 172.21.15.50:0/2938176590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 msgr2=0x55ed6240e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240e3d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd8340986e0 tx=0x7fd834009700 comp rx=0 tx=0).stop 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 -- 172.21.15.50:0/2938176590 shutdown_connections 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6242f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed62404cd0 0x55ed6240e3d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 --2- 172.21.15.50:0/2938176590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed624043b0 0x55ed62406e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 -- 172.21.15.50:0/2938176590 >> 172.21.15.50:0/2938176590 conn(0x55ed62400920 msgr2=0x55ed6240dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 -- 172.21.15.50:0/2938176590 shutdown_connections 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 -- 172.21.15.50:0/2938176590 wait complete. 2024-09-12T03:33:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.426+0000 7fd850e21740 1 -- start start 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 Processor -- start 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 0x55ed6241b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6241bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6241fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ab310 con 0x55ed6240f570 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ac780 con 0x55ed624043b0 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ab9e0 con 0x55ed62404cd0 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.427+0000 7fd850e21740 10 client.0 _ll_register_callbacks cb 0x7fd838040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 0x55ed6241b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 0x55ed6241b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:59990/0 (socket says 172.21.15.50:59990) 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 -- 172.21.15.50:0/612260643 learned_addr learned my addr 172.21.15.50:0/612260643 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847e00640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6241fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd846a00640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6241bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 -- 172.21.15.50:0/612260643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 msgr2=0x55ed6241bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6241bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 -- 172.21.15.50:0/612260643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 msgr2=0x55ed6241fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6241fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 -- 172.21.15.50:0/612260643 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed624202d0 con 0x55ed624043b0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd847400640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 0x55ed6241b570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd83802a4c0 tx=0x7fd838049730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2c5c274dfda71a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd830c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.428+0000 7fd830c00640 1 -- 172.21.15.50:0/612260643 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838060020 con 0x55ed624043b0 2024-09-12T03:33:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8340b1030 con 0x55ed624043b0 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd830c00640 1 -- 172.21.15.50:0/612260643 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fd8380037f0 con 0x55ed624043b0 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd830c00640 1 -- 172.21.15.50:0/612260643 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd83806b050 con 0x55ed624043b0 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 10 client.25718 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ed62492470 con 0x55ed624043b0 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd830c00640 1 -- 172.21.15.50:0/612260643 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8340b1030 con 0x55ed624043b0 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd830c00640 1 -- 172.21.15.50:0/612260643 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd83803c8c0 con 0x55ed624043b0 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 10 client.25718 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 20 client.25718 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62492c10 0x55ed62495030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 4 client.25718 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.429+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed62371b80 con 0x55ed62492c10 2024-09-12T03:33:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.430+0000 7fd846a00640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62492c10 0x55ed62495030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.430+0000 7fd846a00640 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62492c10 0x55ed62495030 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63072b57cb41fb9d server_cookie=40f7183738d26e22 in_seq=0 out_seq=0 2024-09-12T03:33:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.430+0000 7fd830c00640 10 client.25718 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.447+0000 7fd830c00640 1 -- 172.21.15.50:0/612260643 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed62371b80 con 0x55ed62492c10 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.447+0000 7fd830c00640 10 client.25718 handle_command_reply: tid=0 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.447+0000 7fd850e21740 1 client.25718 shutdown 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 msgr2=0x55ed6241b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 0x55ed6241b570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd83802a4c0 tx=0x7fd838049730 comp rx=0 tx=0).stop 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 shutdown_connections 2024-09-12T03:33:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ed6240f570 0x55ed6241fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55ed62492c10 0x55ed62495030 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed62404cd0 0x55ed6241bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 --2- 172.21.15.50:0/612260643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed624043b0 0x55ed6241b570 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd830c00640 0 client.25718 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 >> 172.21.15.50:0/612260643 conn(0x55ed62400920 msgr2=0x55ed623ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 shutdown_connections 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 1 -- 172.21.15.50:0/612260643 wait complete. 2024-09-12T03:33:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.448+0000 7fd850e21740 20 client.25718 trim_cache size 0 max 16384 2024-09-12T03:33:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:33:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:33:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:33:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd82c0a2c80 msgr2=0x7fd82c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd82c0a2c80 0x7fd82c0a5140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8380096c0 tx=0x7fd8380046c0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103400 msgr2=0x7fd848100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103400 0x7fd848100c70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9f0 tx=0x7fd83400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 shutdown_connections 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd82c0a2c80 0x7fd82c0a5140 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd84810bf00 0x7fd8481011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103400 0x7fd848100c70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 --2- 172.21.15.50:0/2789784333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd848102ae0 0x7fd848100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 >> 172.21.15.50:0/2789784333 conn(0x7fd8480777c0 msgr2=0x7fd8480fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 shutdown_connections 2024-09-12T03:33:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:15.450+0000 7fd84ca00640 1 -- 172.21.15.50:0/2789784333 wait complete. 2024-09-12T03:33:16.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:16 smithi050 ceph-mon[30378]: pgmap v186: 129 pgs: 129 active+clean; 105 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.0 MiB/s wr, 361 op/s 2024-09-12T03:33:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:16 smithi142 ceph-mon[35329]: pgmap v186: 129 pgs: 129 active+clean; 105 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.0 MiB/s wr, 361 op/s 2024-09-12T03:33:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:16 smithi082 ceph-mon[35046]: pgmap v186: 129 pgs: 129 active+clean; 105 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.0 MiB/s wr, 361 op/s 2024-09-12T03:33:18.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:18 smithi050 ceph-mon[30378]: pgmap v187: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-09-12T03:33:18.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:18 smithi142 ceph-mon[35329]: pgmap v187: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-09-12T03:33:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:18 smithi082 ceph-mon[35046]: pgmap v187: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-09-12T03:33:20.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:20 smithi050 ceph-mon[30378]: pgmap v188: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.1 MiB/s wr, 379 op/s 2024-09-12T03:33:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:20 smithi142 ceph-mon[35329]: pgmap v188: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.1 MiB/s wr, 379 op/s 2024-09-12T03:33:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:20 smithi082 ceph-mon[35046]: pgmap v188: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.1 MiB/s wr, 379 op/s 2024-09-12T03:33:22.667 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:22 smithi050 ceph-mon[30378]: pgmap v189: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 365 op/s 2024-09-12T03:33:22.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:22 smithi142 ceph-mon[35329]: pgmap v189: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 365 op/s 2024-09-12T03:33:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:22 smithi082 ceph-mon[35046]: pgmap v189: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.1 MiB/s wr, 365 op/s 2024-09-12T03:33:23.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:23 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:23.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:23 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:23.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:23 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:24 smithi050 ceph-mon[30378]: pgmap v190: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 548 op/s 2024-09-12T03:33:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:24 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:24 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:24.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:24 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:33:24.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:24 smithi142 ceph-mon[35329]: pgmap v190: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 548 op/s 2024-09-12T03:33:24.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:24 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:24.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:24 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:24.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:24 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:33:24.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:24 smithi082 ceph-mon[35046]: pgmap v190: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 548 op/s 2024-09-12T03:33:24.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:24 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:24.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:24 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:24.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:24 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:33:26.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:26 smithi050 ceph-mon[30378]: pgmap v191: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 368 op/s 2024-09-12T03:33:26.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:26 smithi142 ceph-mon[35329]: pgmap v191: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 368 op/s 2024-09-12T03:33:26.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:26 smithi082 ceph-mon[35046]: pgmap v191: 129 pgs: 129 active+clean; 109 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 368 op/s 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 -- 172.21.15.50:0/3264153746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781018b0 msgr2=0x7f6e78101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/3264153746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781018b0 0x7f6e78101d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009900 tx=0x7f6e6002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 -- 172.21.15.50:0/3264153746 shutdown_connections 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/3264153746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e78102250 0x7f6e7810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/3264153746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781018b0 0x7f6e78101d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/3264153746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e781078c0 0x7f6e78107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 -- 172.21.15.50:0/3264153746 >> 172.21.15.50:0/3264153746 conn(0x7f6e780fd5d0 msgr2=0x7f6e780ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 -- 172.21.15.50:0/3264153746 shutdown_connections 2024-09-12T03:33:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.597+0000 7f6e7cc00640 1 -- 172.21.15.50:0/3264153746 wait complete. 2024-09-12T03:33:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.598+0000 7f6e7cc00640 1 Processor -- start 2024-09-12T03:33:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.598+0000 7f6e7cc00640 1 -- start start 2024-09-12T03:33:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e7cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 0x7f6e78198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e7cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e78102250 0x7f6e78198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e7cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781078c0 0x7f6e7819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e7cc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78111d20 con 0x7f6e78102250 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e7cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78111ba0 con 0x7f6e781078c0 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e7cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78111ea0 con 0x7f6e781018b0 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e77400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 0x7f6e78198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e78102250 0x7f6e78198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e77400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 0x7f6e78198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:57086/0 (socket says 172.21.15.50:57086) 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e77400640 1 -- 172.21.15.50:0/2924830091 learned_addr learned my addr 172.21.15.50:0/2924830091 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.599+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781078c0 0x7f6e7819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e77400640 1 -- 172.21.15.50:0/2924830091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781078c0 msgr2=0x7f6e7819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e77400640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781078c0 0x7f6e7819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e77400640 1 -- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e78102250 msgr2=0x7f6e78198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e77400640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e78102250 0x7f6e78198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e77400640 1 -- 172.21.15.50:0/2924830091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7819d240 con 0x7f6e781018b0 2024-09-12T03:33:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e77400640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 0x7f6e78198250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e9f0 tx=0x7f6e6800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e74c00640 1 -- 172.21.15.50:0/2924830091 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800ccc0 con 0x7f6e781018b0 2024-09-12T03:33:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e74c00640 1 -- 172.21.15.50:0/2924830091 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e68010910 con 0x7f6e781018b0 2024-09-12T03:33:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e74c00640 1 -- 172.21.15.50:0/2924830091 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680188a0 con 0x7f6e781018b0 2024-09-12T03:33:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.600+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7819d530 con 0x7f6e781018b0 2024-09-12T03:33:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.601+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e781a4dc0 con 0x7f6e781018b0 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.603+0000 7f6e81057740 1 Processor -- start 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 -- start start 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1c97690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1c97bd0 0x555ed1ca6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 0x555ed1ca8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555ed1bfd100 con 0x555ed1c972b0 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e74c00640 1 -- 172.21.15.50:0/2924830091 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6e68010ac0 con 0x7f6e781018b0 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555ed1ba51b0 con 0x555ed1ca6980 2024-09-12T03:33:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e81057740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555ed1bf90c0 con 0x555ed1c97bd0 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 0x555ed1ca8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1c97690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 0x555ed1ca8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57046/0 (socket says 172.21.15.50:57046) 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1239228278 learned_addr learned my addr 172.21.15.50:0/1239228278 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.604+0000 7f6e77400640 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1c97bd0 0x555ed1ca6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1239228278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1c97bd0 msgr2=0x555ed1ca6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1c97bd0 0x555ed1ca6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1239228278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 msgr2=0x555ed1c97690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1c97690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1239228278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed1910210 con 0x555ed1ca6980 2024-09-12T03:33:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 0x555ed1ca8d70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009230 tx=0x7f6e60005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e29c8a5f9c248c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e67e00640 1 -- 172.21.15.50:0/1239228278 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60035040 con 0x555ed1ca6980 2024-09-12T03:33:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e67e00640 1 -- 172.21.15.50:0/1239228278 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e60004400 con 0x555ed1ca6980 2024-09-12T03:33:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.605+0000 7f6e67e00640 1 -- 172.21.15.50:0/1239228278 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600035f0 con 0x555ed1ca6980 2024-09-12T03:33:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e74c00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6e5c0a2d70 0x7f6e5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e74c00640 1 -- 172.21.15.50:0/2924830091 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f6e680cfb60 con 0x7f6e781018b0 2024-09-12T03:33:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e77e00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6e5c0a2d70 0x7f6e5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 -- 172.21.15.50:0/1239228278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 msgr2=0x555ed1ca8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 0x555ed1ca8d70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009230 tx=0x7f6e60005e00 comp rx=0 tx=0).stop 2024-09-12T03:33:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 -- 172.21.15.50:0/1239228278 shutdown_connections 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1ca6980 0x555ed1ca8d70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1c97bd0 0x555ed1ca6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 --2- 172.21.15.50:0/1239228278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1c97690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 -- 172.21.15.50:0/1239228278 >> 172.21.15.50:0/1239228278 conn(0x555ed1c93920 msgr2=0x555ed1c93d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 -- 172.21.15.50:0/1239228278 shutdown_connections 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.606+0000 7f6e81057740 1 -- 172.21.15.50:0/1239228278 wait complete. 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.607+0000 7f6e77e00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6e5c0a2d70 0x7f6e5c0a5230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005fd0 tx=0x7f6e70009500 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.607+0000 7f6e81057740 1 -- start start 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.607+0000 7f6e81057740 1 Processor -- start 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.607+0000 7f6e81057740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1ca3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.607+0000 7f6e81057740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 0x555ed1ca3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e81057740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1ca4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e81057740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555ed1ca9580 con 0x555ed1c972b0 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e81057740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555ed1b86090 con 0x555ed1c97bd0 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e81057740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555ed1ca9700 con 0x555ed1ca6980 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e81057740 10 client.0 _ll_register_callbacks cb 0x555ed1bdaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1ca4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 0x555ed1ca3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1ca4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:57090/0 (socket says 172.21.15.50:57090) 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 0x555ed1ca3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57058/0 (socket says 172.21.15.50:57058) 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e77e00640 1 -- 172.21.15.50:0/1536165896 learned_addr learned my addr 172.21.15.50:0/1536165896 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e77400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1ca3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1536165896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 msgr2=0x555ed1ca4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1ca4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1536165896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 msgr2=0x555ed1ca3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1ca3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.608+0000 7f6e6fe00640 1 -- 172.21.15.50:0/1536165896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed19dc620 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 0x555ed1ca3dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e6002a910 tx=0x7f6e600048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9126b354c3153b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e64c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e64c00640 1 -- 172.21.15.50:0/1536165896 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60035050 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e60004400 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e64c00640 1 -- 172.21.15.50:0/1536165896 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f6e60003800 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e64c00640 1 -- 172.21.15.50:0/1536165896 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e60009520 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e81057740 10 client.25730 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ed1cb5220 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.609+0000 7f6e64c00640 1 -- 172.21.15.50:0/1536165896 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e60004400 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e64c00640 1 -- 172.21.15.50:0/1536165896 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6e60031540 con 0x555ed1c97bd0 2024-09-12T03:33:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e81057740 10 client.25730 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e81057740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e81057740 20 client.25730 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e81057740 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1cb6570 0x555ed1cb8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e81057740 4 client.25730 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ed1c0a830 con 0x555ed1cb6570 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.610+0000 7f6e77400640 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1cb6570 0x555ed1cb8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.611+0000 7f6e77400640 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1cb6570 0x555ed1cb8960 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a269f47daabf9890 server_cookie=89508c866933cac6 in_seq=0 out_seq=0 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.611+0000 7f6e64c00640 10 client.25730 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.611+0000 7f6e64c00640 1 -- 172.21.15.50:0/1536165896 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ed1c0a830 con 0x555ed1cb6570 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.611+0000 7f6e64c00640 10 client.25730 handle_command_reply: tid=0 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 client.25730 shutdown 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 msgr2=0x555ed1ca3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 0x555ed1ca3dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e6002a910 tx=0x7f6e600048f0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 shutdown_connections 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1cb6570 0x555ed1cb8960 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1ca4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c97bd0 0x555ed1ca3dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 --2- 172.21.15.50:0/1536165896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c972b0 0x555ed1ca3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 >> 172.21.15.50:0/1536165896 conn(0x555ed1c93920 msgr2=0x555ed1c99680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e64c00640 0 client.25730 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 shutdown_connections 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 1 -- 172.21.15.50:0/1536165896 wait complete. 2024-09-12T03:33:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.612+0000 7f6e81057740 20 client.25730 trim_cache size 0 max 16384 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.628+0000 7f6e81057740 1 Processor -- start 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.628+0000 7f6e81057740 1 -- start start 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.628+0000 7f6e81057740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cb21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e81057740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cbb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e81057740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e81057740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781af550 con 0x555ed1c97bd0 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e81057740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781ace20 con 0x555ed1c972b0 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e81057740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781aac70 con 0x555ed1ca6980 2024-09-12T03:33:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cb21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57070/0 (socket says 172.21.15.50:57070) 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cbb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 -- 172.21.15.50:0/2654394336 learned_addr learned my addr 172.21.15.50:0/2654394336 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 -- 172.21.15.50:0/2654394336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 msgr2=0x555ed1cbbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 -- 172.21.15.50:0/2654394336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 msgr2=0x555ed1cbb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cbb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 -- 172.21.15.50:0/2654394336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed1ca2810 con 0x555ed1c972b0 2024-09-12T03:33:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.629+0000 7f6e77400640 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cb21e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e6809b990 tx=0x7f6e68009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d2747de0743996 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e67400640 1 -- 172.21.15.50:0/2654394336 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680a0040 con 0x555ed1c972b0 2024-09-12T03:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e67400640 1 -- 172.21.15.50:0/2654394336 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e680a4900 con 0x555ed1c972b0 2024-09-12T03:33:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e67400640 1 -- 172.21.15.50:0/2654394336 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680cd140 con 0x555ed1c972b0 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 -- 172.21.15.50:0/2654394336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 msgr2=0x555ed1cb21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cb21e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e6809b990 tx=0x7f6e68009700 comp rx=0 tx=0).stop 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 -- 172.21.15.50:0/2654394336 shutdown_connections 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cbb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 --2- 172.21.15.50:0/2654394336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cb21e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.630+0000 7f6e81057740 1 -- 172.21.15.50:0/2654394336 >> 172.21.15.50:0/2654394336 conn(0x555ed1c93920 msgr2=0x555ed1c8e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.631+0000 7f6e81057740 1 -- 172.21.15.50:0/2654394336 shutdown_connections 2024-09-12T03:33:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.631+0000 7f6e81057740 1 -- 172.21.15.50:0/2654394336 wait complete. 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.631+0000 7f6e81057740 1 -- start start 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.631+0000 7f6e81057740 1 Processor -- start 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.631+0000 7f6e81057740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cbc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e81057740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cb9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e81057740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cb9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e81057740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781ab2e0 con 0x555ed1c97bd0 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e81057740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781ac750 con 0x555ed1c972b0 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e81057740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781ab9b0 con 0x555ed1ca6980 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e81057740 10 client.0 _ll_register_callbacks cb 0x7f6e70009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cb9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cbc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cbc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:57084/0 (socket says 172.21.15.50:57084) 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 -- 172.21.15.50:0/995423095 learned_addr learned my addr 172.21.15.50:0/995423095 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77e00640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cb9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 -- 172.21.15.50:0/995423095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 msgr2=0x555ed1cb9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cb9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 -- 172.21.15.50:0/995423095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 msgr2=0x555ed1cb9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cb9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e77400640 1 -- 172.21.15.50:0/995423095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed1cba350 con 0x555ed1c972b0 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.632+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cb9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e77400640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cbc1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e680a43d0 tx=0x7f6e680097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d8313b1fb324d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e64c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e64c00640 1 -- 172.21.15.50:0/995423095 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680a0040 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e680a4900 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e64c00640 1 -- 172.21.15.50:0/995423095 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f6e680cd140 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e64c00640 1 -- 172.21.15.50:0/995423095 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e680cd340 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e81057740 10 client.25736 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ed1d20fd0 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.633+0000 7f6e64c00640 1 -- 172.21.15.50:0/995423095 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e680a4900 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e64c00640 1 -- 172.21.15.50:0/995423095 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6e6809ab50 con 0x555ed1c972b0 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e81057740 10 client.25736 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e81057740 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e81057740 20 client.25736 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e81057740 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1d215f0 0x555ed1d239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e81057740 4 client.25736 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ed1c04b80 con 0x555ed1d215f0 2024-09-12T03:33:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1d215f0 0x555ed1d239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e6fe00640 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1d215f0 0x555ed1d239e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67172c31a851cccc server_cookie=e21b4cf154474aba in_seq=0 out_seq=0 2024-09-12T03:33:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.634+0000 7f6e64c00640 10 client.25736 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e64c00640 1 -- 172.21.15.50:0/995423095 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ed1c04b80 con 0x555ed1d215f0 2024-09-12T03:33:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e64c00640 10 client.25736 handle_command_reply: tid=0 2024-09-12T03:33:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 client.25736 shutdown 2024-09-12T03:33:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 msgr2=0x555ed1cbc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cbc1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e680a43d0 tx=0x7f6e680097a0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 shutdown_connections 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x555ed1d215f0 0x555ed1d239e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ed1ca6980 0x555ed1cb9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ed1c97bd0 0x555ed1cb9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e64c00640 0 client.25736 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 --2- 172.21.15.50:0/995423095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ed1c972b0 0x555ed1cbc1f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 >> 172.21.15.50:0/995423095 conn(0x555ed1c93920 msgr2=0x555ed1c929c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 shutdown_connections 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.648+0000 7f6e81057740 1 -- 172.21.15.50:0/995423095 wait complete. 2024-09-12T03:33:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.649+0000 7f6e81057740 20 client.25736 trim_cache size 0 max 16384 2024-09-12T03:33:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:33:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:33:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:33:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.649+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6e5c0a2d70 msgr2=0x7f6e5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.649+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6e5c0a2d70 0x7f6e5c0a5230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005fd0 tx=0x7f6e70009500 comp rx=0 tx=0).stop 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.649+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 msgr2=0x7f6e78198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.649+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 0x7f6e78198250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e9f0 tx=0x7f6e6800edf0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 shutdown_connections 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e781078c0 0x7f6e7819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6e5c0a2d70 0x7f6e5c0a5230 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e78102250 0x7f6e78198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 --2- 172.21.15.50:0/2924830091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e781018b0 0x7f6e78198250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 >> 172.21.15.50:0/2924830091 conn(0x7f6e780fd5d0 msgr2=0x7f6e780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 shutdown_connections 2024-09-12T03:33:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:28.650+0000 7f6e7cc00640 1 -- 172.21.15.50:0/2924830091 wait complete. 2024-09-12T03:33:28.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:28 smithi050 ceph-mon[30378]: pgmap v192: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 535 op/s 2024-09-12T03:33:28.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:28 smithi142 ceph-mon[35329]: pgmap v192: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 535 op/s 2024-09-12T03:33:28.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:28 smithi082 ceph-mon[35046]: pgmap v192: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 535 op/s 2024-09-12T03:33:30.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:30 smithi050 ceph-mon[30378]: pgmap v193: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-09-12T03:33:30.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:30 smithi142 ceph-mon[35329]: pgmap v193: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-09-12T03:33:30.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:30 smithi082 ceph-mon[35046]: pgmap v193: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-09-12T03:33:32.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:32 smithi050 ceph-mon[30378]: pgmap v194: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 364 op/s 2024-09-12T03:33:32.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:32 smithi142 ceph-mon[35329]: pgmap v194: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 364 op/s 2024-09-12T03:33:32.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:32 smithi082 ceph-mon[35046]: pgmap v194: 129 pgs: 129 active+clean; 117 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 364 op/s 2024-09-12T03:33:34.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:34 smithi050 ceph-mon[30378]: pgmap v195: 129 pgs: 129 active+clean; 103 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 550 op/s 2024-09-12T03:33:34.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:34 smithi142 ceph-mon[35329]: pgmap v195: 129 pgs: 129 active+clean; 103 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 550 op/s 2024-09-12T03:33:34.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:34 smithi082 ceph-mon[35046]: pgmap v195: 129 pgs: 129 active+clean; 103 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 550 op/s 2024-09-12T03:33:36.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:36 smithi050 ceph-mon[30378]: pgmap v196: 129 pgs: 129 active+clean; 103 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.0 MiB/s wr, 367 op/s 2024-09-12T03:33:36.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:36 smithi142 ceph-mon[35329]: pgmap v196: 129 pgs: 129 active+clean; 103 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.0 MiB/s wr, 367 op/s 2024-09-12T03:33:36.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:36 smithi082 ceph-mon[35046]: pgmap v196: 129 pgs: 129 active+clean; 103 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.0 MiB/s wr, 367 op/s 2024-09-12T03:33:38.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:38 smithi050 ceph-mon[30378]: pgmap v197: 129 pgs: 129 active+clean; 107 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 524 op/s 2024-09-12T03:33:38.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:38 smithi142 ceph-mon[35329]: pgmap v197: 129 pgs: 129 active+clean; 107 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 524 op/s 2024-09-12T03:33:38.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:38 smithi082 ceph-mon[35046]: pgmap v197: 129 pgs: 129 active+clean; 107 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 524 op/s 2024-09-12T03:33:40.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:40 smithi142 ceph-mon[35329]: pgmap v198: 129 pgs: 129 active+clean; 109 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-09-12T03:33:40.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:40 smithi082 ceph-mon[35046]: pgmap v198: 129 pgs: 129 active+clean; 109 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-09-12T03:33:40.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:40 smithi050 ceph-mon[30378]: pgmap v198: 129 pgs: 129 active+clean; 109 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-09-12T03:33:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 -- 172.21.15.50:0/2862112108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc68010e3d0 msgr2=0x7fc6801107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 --2- 172.21.15.50:0/2862112108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc68010e3d0 0x7fc6801107c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc678009be0 tx=0x7fc67802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 -- 172.21.15.50:0/2862112108 shutdown_connections 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 --2- 172.21.15.50:0/2862112108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc68010e3d0 0x7fc6801107c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 --2- 172.21.15.50:0/2862112108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc680101140 0x7fc68010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 --2- 172.21.15.50:0/2862112108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100820 0x7fc680100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 -- 172.21.15.50:0/2862112108 >> 172.21.15.50:0/2862112108 conn(0x7fc6800fc6f0 msgr2=0x7fc6800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.806+0000 7fc688000640 1 -- 172.21.15.50:0/2862112108 shutdown_connections 2024-09-12T03:33:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.807+0000 7fc688000640 1 -- 172.21.15.50:0/2862112108 wait complete. 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.807+0000 7fc688000640 1 Processor -- start 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 -- start start 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100820 0x7fc68019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc680101140 0x7fc68019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 0x7fc68019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114090 con 0x7fc680100820 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc680113f10 con 0x7fc68010e3d0 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.808+0000 7fc688000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc680114210 con 0x7fc680101140 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 0x7fc68019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 0x7fc68019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42992/0 (socket says 172.21.15.50:42992) 2024-09-12T03:33:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 -- 172.21.15.50:0/26704171 learned_addr learned my addr 172.21.15.50:0/26704171 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc684c00640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc680101140 0x7fc68019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc685600640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100820 0x7fc68019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 -- 172.21.15.50:0/26704171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc680101140 msgr2=0x7fc68019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc680101140 0x7fc68019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 -- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100820 msgr2=0x7fc68019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100820 0x7fc68019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.809+0000 7fc686000640 1 -- 172.21.15.50:0/26704171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68019f4d0 con 0x7fc68010e3d0 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.810+0000 7fc686000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 0x7fc68019eda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc67802abe0 tx=0x7fc6780305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.810+0000 7fc67e000640 1 -- 172.21.15.50:0/26704171 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678041020 con 0x7fc68010e3d0 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.810+0000 7fc67e000640 1 -- 172.21.15.50:0/26704171 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc678003e50 con 0x7fc68010e3d0 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.810+0000 7fc67e000640 1 -- 172.21.15.50:0/26704171 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678036040 con 0x7fc68010e3d0 2024-09-12T03:33:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.810+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68019f760 con 0x7fc68010e3d0 2024-09-12T03:33:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.811+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6801a7050 con 0x7fc68010e3d0 2024-09-12T03:33:41.815 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.815+0000 7fc67e000640 1 -- 172.21.15.50:0/26704171 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc678044020 con 0x7fc68010e3d0 2024-09-12T03:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 Processor -- start 2024-09-12T03:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 -- start start 2024-09-12T03:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed2495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed253e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2543a0 0x5590ed256790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5590ed1af100 con 0x5590ed249b00 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5590ed1571b0 con 0x5590ed2491e0 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.816+0000 7fc68afbe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5590ed1ab0c0 con 0x5590ed2543a0 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed2495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed2495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42998/0 (socket says 172.21.15.50:42998) 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 -- 172.21.15.50:0/1814985275 learned_addr learned my addr 172.21.15.50:0/1814985275 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2543a0 0x5590ed256790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc685600640 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed253e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc67e000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc6680a2ea0 0x7fc6680a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 -- 172.21.15.50:0/1814985275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 msgr2=0x5590ed2495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed2495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 -- 172.21.15.50:0/1814985275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 msgr2=0x5590ed253e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed253e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc684c00640 1 -- 172.21.15.50:0/1814985275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ecec2210 con 0x5590ed2543a0 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.817+0000 7fc67e000640 1 -- 172.21.15.50:0/26704171 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fc6780efee0 con 0x7fc68010e3d0 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.818+0000 7fc686000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc6680a2ea0 0x7fc6680a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.818+0000 7fc684c00640 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2543a0 0x5590ed256790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0127a0 tx=0x7fc66c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7335e533e20c7bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.818+0000 7fc672600640 1 -- 172.21.15.50:0/1814985275 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c012050 con 0x5590ed2543a0 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.818+0000 7fc672600640 1 -- 172.21.15.50:0/1814985275 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c0121f0 con 0x5590ed2543a0 2024-09-12T03:33:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.818+0000 7fc672600640 1 -- 172.21.15.50:0/1814985275 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0035f0 con 0x5590ed2543a0 2024-09-12T03:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 -- 172.21.15.50:0/1814985275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2543a0 msgr2=0x5590ed256790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc686000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc6680a2ea0 0x7fc6680a5360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc6780ed2a0 tx=0x7fc678009280 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2543a0 0x5590ed256790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0127a0 tx=0x7fc66c005f90 comp rx=0 tx=0).stop 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 -- 172.21.15.50:0/1814985275 shutdown_connections 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2543a0 0x5590ed256790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed253e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 --2- 172.21.15.50:0/1814985275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed2495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 -- 172.21.15.50:0/1814985275 >> 172.21.15.50:0/1814985275 conn(0x5590ed245750 msgr2=0x5590ed245b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 -- 172.21.15.50:0/1814985275 shutdown_connections 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.819+0000 7fc68afbe740 1 -- 172.21.15.50:0/1814985275 wait complete. 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 -- start start 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 Processor -- start 2024-09-12T03:33:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2491e0 0x5590ed2513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed251930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2543a0 0x5590ed251e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5590ed256f70 con 0x5590ed249b00 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5590ed138090 con 0x5590ed2543a0 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.820+0000 7fc68afbe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5590ed2570f0 con 0x5590ed2491e0 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc68afbe740 10 client.0 _ll_register_callbacks cb 0x5590ed18cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed251930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc684c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2491e0 0x5590ed2513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed251930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59236/0 (socket says 172.21.15.50:59236) 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 -- 172.21.15.50:0/491344305 learned_addr learned my addr 172.21.15.50:0/491344305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc684c00640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2543a0 0x5590ed251e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 -- 172.21.15.50:0/491344305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2491e0 msgr2=0x5590ed2513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2491e0 0x5590ed2513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 -- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2543a0 msgr2=0x5590ed251e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2543a0 0x5590ed251e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc685600640 1 -- 172.21.15.50:0/491344305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ecf8e620 con 0x5590ed249b00 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.821+0000 7fc684c00640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2543a0 0x5590ed251e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc685600640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed251930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc6740126d0 tx=0x7fc674012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878b1fe35f8b7c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc666a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc666a00640 1 -- 172.21.15.50:0/491344305 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674011bb0 con 0x5590ed249b00 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc666a00640 1 -- 172.21.15.50:0/491344305 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fc674011d50 con 0x5590ed249b00 2024-09-12T03:33:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc66c0121f0 con 0x5590ed249b00 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc666a00640 1 -- 172.21.15.50:0/491344305 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc674004030 con 0x5590ed249b00 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc68afbe740 10 client.15711 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590ed262cc0 con 0x5590ed249b00 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.822+0000 7fc666a00640 1 -- 172.21.15.50:0/491344305 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c0121f0 con 0x5590ed249b00 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc666a00640 1 -- 172.21.15.50:0/491344305 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc67401c460 con 0x5590ed249b00 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc68afbe740 10 client.15711 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc68afbe740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc68afbe740 20 client.15711 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc68afbe740 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2641d0 0x5590ed2665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc68afbe740 4 client.15711 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590ed1bc830 con 0x5590ed2641d0 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.823+0000 7fc684c00640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2641d0 0x5590ed2665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.824+0000 7fc684c00640 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2641d0 0x5590ed2665c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea243834376f14f server_cookie=dfe84901968496be in_seq=0 out_seq=0 2024-09-12T03:33:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.824+0000 7fc666a00640 10 client.15711 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.824+0000 7fc666a00640 1 -- 172.21.15.50:0/491344305 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590ed1bc830 con 0x5590ed2641d0 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.824+0000 7fc666a00640 10 client.15711 handle_command_reply: tid=0 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.824+0000 7fc68afbe740 1 client.15711 shutdown 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 msgr2=0x5590ed251930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed251930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc6740126d0 tx=0x7fc674012b90 comp rx=0 tx=0).stop 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 shutdown_connections 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2543a0 0x5590ed251e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed249b00 0x5590ed251930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2641d0 0x5590ed2665c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 --2- 172.21.15.50:0/491344305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed2491e0 0x5590ed2513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 >> 172.21.15.50:0/491344305 conn(0x5590ed245750 msgr2=0x5590ed24b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc666a00640 0 client.15711 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 shutdown_connections 2024-09-12T03:33:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 1 -- 172.21.15.50:0/491344305 wait complete. 2024-09-12T03:33:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.825+0000 7fc68afbe740 20 client.15711 trim_cache size 0 max 16384 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 Processor -- start 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 -- start start 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed251cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed26be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed26c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b3a60 con 0x5590ed2543a0 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b1330 con 0x5590ed2491e0 2024-09-12T03:33:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc68afbe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801af180 con 0x5590ed249b00 2024-09-12T03:33:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc685600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed26be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc685600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed26be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:57000/0 (socket says 172.21.15.50:57000) 2024-09-12T03:33:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.845+0000 7fc685600640 1 -- 172.21.15.50:0/3857844565 learned_addr learned my addr 172.21.15.50:0/3857844565 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc684c00640 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed251cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc684c00640 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed26c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc685600640 1 -- 172.21.15.50:0/3857844565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 msgr2=0x5590ed251cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc685600640 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed251cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc685600640 1 -- 172.21.15.50:0/3857844565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 msgr2=0x5590ed26c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc685600640 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed26c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.846+0000 7fc685600640 1 -- 172.21.15.50:0/3857844565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ed249010 con 0x5590ed249b00 2024-09-12T03:33:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.847+0000 7fc685600640 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed26be80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc67401ca70 tx=0x7fc674005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29afa2e56d2134e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.847+0000 7fc671c00640 1 -- 172.21.15.50:0/3857844565 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67402e020 con 0x5590ed249b00 2024-09-12T03:33:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.847+0000 7fc671c00640 1 -- 172.21.15.50:0/3857844565 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67401c460 con 0x5590ed249b00 2024-09-12T03:33:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.847+0000 7fc671c00640 1 -- 172.21.15.50:0/3857844565 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674005460 con 0x5590ed249b00 2024-09-12T03:33:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.852+0000 7fc68afbe740 1 -- 172.21.15.50:0/3857844565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 msgr2=0x5590ed26be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.852+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed26be80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc67401ca70 tx=0x7fc674005a00 comp rx=0 tx=0).stop 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.853+0000 7fc68afbe740 1 -- 172.21.15.50:0/3857844565 shutdown_connections 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.853+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed26c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.853+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed26be80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.853+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3857844565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed251cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.853+0000 7fc68afbe740 1 -- 172.21.15.50:0/3857844565 >> 172.21.15.50:0/3857844565 conn(0x5590ed245750 msgr2=0x5590ed24fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.854+0000 7fc68afbe740 1 -- 172.21.15.50:0/3857844565 shutdown_connections 2024-09-12T03:33:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.854+0000 7fc68afbe740 1 -- 172.21.15.50:0/3857844565 wait complete. 2024-09-12T03:33:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.854+0000 7fc68afbe740 1 -- start start 2024-09-12T03:33:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.855+0000 7fc68afbe740 1 Processor -- start 2024-09-12T03:33:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed25c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed25cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed24de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801af7f0 con 0x5590ed2543a0 2024-09-12T03:33:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801b0c60 con 0x5590ed2491e0 2024-09-12T03:33:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801afec0 con 0x5590ed249b00 2024-09-12T03:33:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.856+0000 7fc68afbe740 10 client.0 _ll_register_callbacks cb 0x7fc674005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.857+0000 7fc685600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed25cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.857+0000 7fc685600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed25cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:57006/0 (socket says 172.21.15.50:57006) 2024-09-12T03:33:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.857+0000 7fc685600640 1 -- 172.21.15.50:0/3061232905 learned_addr learned my addr 172.21.15.50:0/3061232905 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.857+0000 7fc684c00640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed24de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.858+0000 7fc684c00640 1 -- 172.21.15.50:0/3061232905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 msgr2=0x5590ed25cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.858+0000 7fc684c00640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed25cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.859+0000 7fc684c00640 1 -- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 msgr2=0x5590ed25c650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:33:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.859+0000 7fc684c00640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed25c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.859+0000 7fc684c00640 1 -- 172.21.15.50:0/3061232905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ed24e610 con 0x5590ed2543a0 2024-09-12T03:33:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.859+0000 7fc685600640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed25cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.860+0000 7fc684c00640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed24de10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc66c01c040 tx=0x7fc66c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5533e4b02f329dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.860+0000 7fc666a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.861+0000 7fc666a00640 1 -- 172.21.15.50:0/3061232905 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c033060 con 0x5590ed2543a0 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.861+0000 7fc666a00640 1 -- 172.21.15.50:0/3061232905 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fc66c00dd60 con 0x5590ed2543a0 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.861+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67401c460 con 0x5590ed2543a0 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.861+0000 7fc666a00640 1 -- 172.21.15.50:0/3061232905 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc66c00d710 con 0x5590ed2543a0 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.861+0000 7fc68afbe740 10 client.15717 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.861+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590ed2ce9e0 con 0x5590ed2543a0 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.862+0000 7fc666a00640 1 -- 172.21.15.50:0/3061232905 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67401c460 con 0x5590ed2543a0 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.862+0000 7fc666a00640 1 -- 172.21.15.50:0/3061232905 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc66c02e030 con 0x5590ed2543a0 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.862+0000 7fc68afbe740 10 client.15717 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.862+0000 7fc68afbe740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.862+0000 7fc68afbe740 20 client.15717 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.863+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2ced60 0x5590ed2d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.863+0000 7fc68afbe740 4 client.15717 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.863+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590ed1b6b80 con 0x5590ed2ced60 2024-09-12T03:33:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.863+0000 7fc685600640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2ced60 0x5590ed2d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.864+0000 7fc685600640 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2ced60 0x5590ed2d1150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7ad419f60dffc0 server_cookie=1a271e2913be0f54 in_seq=0 out_seq=0 2024-09-12T03:33:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.864+0000 7fc666a00640 10 client.15717 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.879+0000 7fc666a00640 1 -- 172.21.15.50:0/3061232905 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590ed1b6b80 con 0x5590ed2ced60 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc666a00640 10 client.15717 handle_command_reply: tid=0 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 client.15717 shutdown 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 msgr2=0x5590ed24de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed24de10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc66c01c040 tx=0x7fc66c001e50 comp rx=0 tx=0).stop 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 shutdown_connections 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5590ed2543a0 0x5590ed24de10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5590ed2ced60 0x5590ed2d1150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ed249b00 0x5590ed25cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc666a00640 0 client.15717 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 --2- 172.21.15.50:0/3061232905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ed2491e0 0x5590ed25c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 >> 172.21.15.50:0/3061232905 conn(0x5590ed245750 msgr2=0x5590ed25f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 shutdown_connections 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 1 -- 172.21.15.50:0/3061232905 wait complete. 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.880+0000 7fc68afbe740 20 client.15717 trim_cache size 0 max 16384 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:33:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:33:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc6680a2ea0 msgr2=0x7fc6680a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc6680a2ea0 0x7fc6680a5360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc6780ed2a0 tx=0x7fc678009280 comp rx=0 tx=0).stop 2024-09-12T03:33:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 msgr2=0x7fc68019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 0x7fc68019eda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc67802abe0 tx=0x7fc6780305d0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 shutdown_connections 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc68010e3d0 0x7fc68019eda0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc680101140 0x7fc68019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc6680a2ea0 0x7fc6680a5360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 --2- 172.21.15.50:0/26704171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100820 0x7fc68019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 >> 172.21.15.50:0/26704171 conn(0x7fc6800fc6f0 msgr2=0x7fc6800feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 shutdown_connections 2024-09-12T03:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:41.881+0000 7fc688000640 1 -- 172.21.15.50:0/26704171 wait complete. 2024-09-12T03:33:42.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:42 smithi142 ceph-mon[35329]: pgmap v199: 129 pgs: 129 active+clean; 109 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-09-12T03:33:42.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:42 smithi082 ceph-mon[35046]: pgmap v199: 129 pgs: 129 active+clean; 109 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-09-12T03:33:42.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:42 smithi050 ceph-mon[30378]: pgmap v199: 129 pgs: 129 active+clean; 109 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-09-12T03:33:44.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:44 smithi142 ceph-mon[35329]: pgmap v200: 129 pgs: 129 active+clean; 105 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.5 MiB/s wr, 531 op/s 2024-09-12T03:33:44.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:44 smithi082 ceph-mon[35046]: pgmap v200: 129 pgs: 129 active+clean; 105 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.5 MiB/s wr, 531 op/s 2024-09-12T03:33:44.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:44 smithi050 ceph-mon[30378]: pgmap v200: 129 pgs: 129 active+clean; 105 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.5 MiB/s wr, 531 op/s 2024-09-12T03:33:46.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:46 smithi142 ceph-mon[35329]: pgmap v201: 129 pgs: 129 active+clean; 105 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.0 MiB/s wr, 344 op/s 2024-09-12T03:33:46.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:46 smithi082 ceph-mon[35046]: pgmap v201: 129 pgs: 129 active+clean; 105 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.0 MiB/s wr, 344 op/s 2024-09-12T03:33:46.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:46 smithi050 ceph-mon[30378]: pgmap v201: 129 pgs: 129 active+clean; 105 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.0 MiB/s wr, 344 op/s 2024-09-12T03:33:48.715 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:48 smithi050 ceph-mon[30378]: pgmap v202: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2024-09-12T03:33:48.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:48 smithi142 ceph-mon[35329]: pgmap v202: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2024-09-12T03:33:48.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:48 smithi082 ceph-mon[35046]: pgmap v202: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2024-09-12T03:33:50.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:50 smithi142 ceph-mon[35329]: pgmap v203: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 359 op/s 2024-09-12T03:33:50.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:50 smithi082 ceph-mon[35046]: pgmap v203: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 359 op/s 2024-09-12T03:33:50.917 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:50 smithi050 ceph-mon[30378]: pgmap v203: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 359 op/s 2024-09-12T03:33:52.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:52 smithi142 ceph-mon[35329]: pgmap v204: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.0 MiB/s wr, 344 op/s 2024-09-12T03:33:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:52 smithi082 ceph-mon[35046]: pgmap v204: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.0 MiB/s wr, 344 op/s 2024-09-12T03:33:52.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:52 smithi050 ceph-mon[30378]: pgmap v204: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.0 MiB/s wr, 344 op/s 2024-09-12T03:33:53.832 INFO:tasks.workunit.client.0.smithi050.stdout:All 50000 operations completed A-OK! 2024-09-12T03:33:53.870 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:33:53.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:33:54.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:54 smithi142 ceph-mon[35329]: pgmap v205: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 518 op/s 2024-09-12T03:33:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:54 smithi082 ceph-mon[35046]: pgmap v205: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 518 op/s 2024-09-12T03:33:54.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:54 smithi050 ceph-mon[30378]: pgmap v205: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 518 op/s 2024-09-12T03:33:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 -- 172.21.15.50:0/1437272540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 msgr2=0x7f8434110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 --2- 172.21.15.50:0/1437272540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f8434110790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f842c009be0 tx=0x7f842c02a750 comp rx=0 tx=0).stop 2024-09-12T03:33:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 -- 172.21.15.50:0/1437272540 shutdown_connections 2024-09-12T03:33:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 --2- 172.21.15.50:0/1437272540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f8434110790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 --2- 172.21.15.50:0/1437272540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8434074ec0 0x7f843410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 --2- 172.21.15.50:0/1437272540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84340745a0 0x7f8434074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.001+0000 7f843c200640 1 -- 172.21.15.50:0/1437272540 >> 172.21.15.50:0/1437272540 conn(0x7f84340fd7e0 msgr2=0x7f84340ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.002+0000 7f843c200640 1 -- 172.21.15.50:0/1437272540 shutdown_connections 2024-09-12T03:33:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.002+0000 7f843c200640 1 -- 172.21.15.50:0/1437272540 wait complete. 2024-09-12T03:33:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.003+0000 7f843c200640 1 Processor -- start 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.003+0000 7f843c200640 1 -- start start 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.004+0000 7f843c200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84340745a0 0x7f843410dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.004+0000 7f843c200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8434074ec0 0x7f843410ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.004+0000 7f843c200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f843410b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.004+0000 7f843c200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8434114090 con 0x7f8434074ec0 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.004+0000 7f843c200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8434113f10 con 0x7f843410e3a0 2024-09-12T03:33:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.004+0000 7f843c200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8434114210 con 0x7f84340745a0 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f843410b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f843410b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34134/0 (socket says 172.21.15.50:34134) 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 -- 172.21.15.50:0/1930127837 learned_addr learned my addr 172.21.15.50:0/1930127837 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 -- 172.21.15.50:0/1930127837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84340745a0 msgr2=0x7f843410dcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84340745a0 0x7f843410dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 -- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8434074ec0 msgr2=0x7f843410ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8434074ec0 0x7f843410ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.005+0000 7f843a200640 1 -- 172.21.15.50:0/1930127837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843410bb00 con 0x7f843410e3a0 2024-09-12T03:33:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.006+0000 7f843a200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f843410b290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f842c009b30 tx=0x7f842c0305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.007+0000 7f8432000640 1 -- 172.21.15.50:0/1930127837 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c041020 con 0x7f843410e3a0 2024-09-12T03:33:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.007+0000 7f8432000640 1 -- 172.21.15.50:0/1930127837 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842c003e50 con 0x7f843410e3a0 2024-09-12T03:33:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.007+0000 7f8432000640 1 -- 172.21.15.50:0/1930127837 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c036040 con 0x7f843410e3a0 2024-09-12T03:33:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.007+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341995c0 con 0x7f843410e3a0 2024-09-12T03:33:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.008+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8434199a30 con 0x7f843410e3a0 2024-09-12T03:33:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.012+0000 7f8432000640 1 -- 172.21.15.50:0/1930127837 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f842c044020 con 0x7f843410e3a0 2024-09-12T03:33:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 Processor -- start 2024-09-12T03:33:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 -- start start 2024-09-12T03:33:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c06655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c0672790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a3c05cb100 con 0x55a3c06703a0 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a3c05731b0 con 0x55a3c06651e0 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f843f027740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a3c05c70c0 con 0x55a3c0665b00 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f8432000640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f841c0a2f00 0x7f841c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.013+0000 7f8438e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c06655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8439800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c06655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:34146/0 (socket says 172.21.15.50:34146) 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 -- 172.21.15.50:0/1391830306 learned_addr learned my addr 172.21.15.50:0/1391830306 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8432000640 1 -- 172.21.15.50:0/1930127837 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f842c0f0540 con 0x7f843410e3a0 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c0672790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 -- 172.21.15.50:0/1391830306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 msgr2=0x55a3c066fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 -- 172.21.15.50:0/1391830306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 msgr2=0x55a3c0672790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c0672790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 -- 172.21.15.50:0/1391830306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c02de210 con 0x55a3c06651e0 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8439800640 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f8438e00640 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c06655c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8420009fc0 tx=0x7f8420009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1a854147de289c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.014+0000 7f843a200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f841c0a2f00 0x7f841c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.015+0000 7f8426a00640 1 -- 172.21.15.50:0/1391830306 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420010e80 con 0x55a3c06651e0 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.015+0000 7f8426a00640 1 -- 172.21.15.50:0/1391830306 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8420005020 con 0x55a3c06651e0 2024-09-12T03:33:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.015+0000 7f8426a00640 1 -- 172.21.15.50:0/1391830306 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420004990 con 0x55a3c06651e0 2024-09-12T03:33:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.015+0000 7f843f027740 1 -- 172.21.15.50:0/1391830306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 msgr2=0x55a3c06655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.015+0000 7f843a200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f841c0a2f00 0x7f841c0a53c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f842c0c4db0 tx=0x7f842c02a750 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.015+0000 7f843f027740 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c06655c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8420009fc0 tx=0x7f8420009610 comp rx=0 tx=0).stop 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 -- 172.21.15.50:0/1391830306 shutdown_connections 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c0672790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 --2- 172.21.15.50:0/1391830306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c06655c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 -- 172.21.15.50:0/1391830306 >> 172.21.15.50:0/1391830306 conn(0x55a3c0661750 msgr2=0x55a3c0661b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 -- 172.21.15.50:0/1391830306 shutdown_connections 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 -- 172.21.15.50:0/1391830306 wait complete. 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 -- start start 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.016+0000 7f843f027740 1 Processor -- start 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c066d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c066dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a3c0672f70 con 0x55a3c06703a0 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a3c0554090 con 0x55a3c06651e0 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a3c06730f0 con 0x55a3c0665b00 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f843f027740 10 client.0 _ll_register_callbacks cb 0x55a3c05a8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:59294/0 (socket says 172.21.15.50:59294) 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 -- 172.21.15.50:0/536631400 learned_addr learned my addr 172.21.15.50:0/536631400 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8438e00640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c066d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8438e00640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c066dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 -- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 msgr2=0x55a3c066d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c066d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 -- 172.21.15.50:0/536631400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 msgr2=0x55a3c066dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c066dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8439800640 1 -- 172.21.15.50:0/536631400 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c03aa620 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.017+0000 7f8438e00640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c066dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f8439800640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066d820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8428011720 tx=0x7f8428011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=965cebbd037fffd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f841b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f841b400640 1 -- 172.21.15.50:0/536631400 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842801e060 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8420005020 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f841b400640 1 -- 172.21.15.50:0/536631400 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f842800fd30 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f841b400640 1 -- 172.21.15.50:0/536631400 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8428019430 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f843f027740 10 client.25744 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.018+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3c067ec20 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f841b400640 1 -- 172.21.15.50:0/536631400 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8420005020 con 0x55a3c0665b00 2024-09-12T03:33:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f841b400640 1 -- 172.21.15.50:0/536631400 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f84280224c0 con 0x55a3c0665b00 2024-09-12T03:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f843f027740 10 client.25744 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f843f027740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f843f027740 20 client.25744 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f843f027740 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c0680090 0x55a3c0682480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f843f027740 4 client.25744 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.019+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c05d8830 con 0x55a3c0680090 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.020+0000 7f8438e00640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c0680090 0x55a3c0682480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.020+0000 7f8438e00640 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c0680090 0x55a3c0682480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59875677225f877a server_cookie=66d9d107b0eef469 in_seq=0 out_seq=0 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.020+0000 7f841b400640 10 client.25744 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f841b400640 1 -- 172.21.15.50:0/536631400 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3c05d8830 con 0x55a3c0680090 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f841b400640 10 client.25744 handle_command_reply: tid=0 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 client.25744 shutdown 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 msgr2=0x55a3c066d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066d820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8428011720 tx=0x7f8428011be0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 shutdown_connections 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06703a0 0x55a3c066dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c0680090 0x55a3c0682480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c066d820 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 --2- 172.21.15.50:0/536631400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06651e0 0x55a3c066d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f841b400640 0 client.25744 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 >> 172.21.15.50:0/536631400 conn(0x55a3c0661750 msgr2=0x55a3c0667460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 shutdown_connections 2024-09-12T03:33:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 1 -- 172.21.15.50:0/536631400 wait complete. 2024-09-12T03:33:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.021+0000 7f843f027740 20 client.25744 trim_cache size 0 max 16384 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.036+0000 7f843f027740 1 Processor -- start 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 -- start start 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 0x55a3c0687d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c0665b00 0x55a3c0688260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0666c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84341c0650 con 0x55a3c0665b00 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84341bdf20 con 0x55a3c06703a0 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f843f027740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f843410c280 con 0x55a3c06651e0 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f8438e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 0x55a3c0687d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f8439800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c0665b00 0x55a3c0688260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f8438e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 0x55a3c0687d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:59302/0 (socket says 172.21.15.50:59302) 2024-09-12T03:33:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f8438e00640 1 -- 172.21.15.50:0/2402916402 learned_addr learned my addr 172.21.15.50:0/2402916402 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.037+0000 7f8438e00640 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0666c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8438e00640 1 -- 172.21.15.50:0/2402916402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 msgr2=0x55a3c0666c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8438e00640 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0666c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8438e00640 1 -- 172.21.15.50:0/2402916402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c0665b00 msgr2=0x55a3c0688260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8438e00640 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c0665b00 0x55a3c0688260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8438e00640 1 -- 172.21.15.50:0/2402916402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c0665930 con 0x55a3c06651e0 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8439800640 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c0665b00 0x55a3c0688260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8438e00640 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 0x55a3c0687d20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84200095e0 tx=0x7f8420014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7da08259731b437a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8426000640 1 -- 172.21.15.50:0/2402916402 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420004240 con 0x55a3c06651e0 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8426000640 1 -- 172.21.15.50:0/2402916402 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84200043e0 con 0x55a3c06651e0 2024-09-12T03:33:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.038+0000 7f8426000640 1 -- 172.21.15.50:0/2402916402 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842002a7f0 con 0x55a3c06651e0 2024-09-12T03:33:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 -- 172.21.15.50:0/2402916402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 msgr2=0x55a3c0687d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 0x55a3c0687d20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84200095e0 tx=0x7f8420014040 comp rx=0 tx=0).stop 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 -- 172.21.15.50:0/2402916402 shutdown_connections 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0666c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c0665b00 0x55a3c0688260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 --2- 172.21.15.50:0/2402916402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c06651e0 0x55a3c0687d20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 -- 172.21.15.50:0/2402916402 >> 172.21.15.50:0/2402916402 conn(0x55a3c0661750 msgr2=0x55a3c0661b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 -- 172.21.15.50:0/2402916402 shutdown_connections 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.039+0000 7f843f027740 1 -- 172.21.15.50:0/2402916402 wait complete. 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 -- start start 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 Processor -- start 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 0x55a3c0678650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c0678b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0669d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84341a1aa0 con 0x55a3c06651e0 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84341bd850 con 0x55a3c06703a0 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.040+0000 7f843f027740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84341bcab0 con 0x55a3c0665b00 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f843f027740 10 client.0 _ll_register_callbacks cb 0x7f8428009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8438e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 0x55a3c0678650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8438e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 0x55a3c0678650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50362/0 (socket says 172.21.15.50:50362) 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8438e00640 1 -- 172.21.15.50:0/3451667768 learned_addr learned my addr 172.21.15.50:0/3451667768 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8439800640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c0678b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8438e00640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0669d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8439800640 1 -- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 msgr2=0x55a3c0669d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8439800640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0669d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8439800640 1 -- 172.21.15.50:0/3451667768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 msgr2=0x55a3c0678650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8439800640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 0x55a3c0678650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8439800640 1 -- 172.21.15.50:0/3451667768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c066a5a0 con 0x55a3c0665b00 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.041+0000 7f8438e00640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 0x55a3c0678650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f8439800640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c0678b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8428022a00 tx=0x7f8428022d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af501770a929e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f841b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f841b400640 1 -- 172.21.15.50:0/3451667768 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428030060 con 0x55a3c0665b00 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f841b400640 1 -- 172.21.15.50:0/3451667768 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7f84280225b0 con 0x55a3c0665b00 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84200043e0 con 0x55a3c0665b00 2024-09-12T03:33:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f841b400640 1 -- 172.21.15.50:0/3451667768 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8428037050 con 0x55a3c0665b00 2024-09-12T03:33:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f843f027740 10 client.25750 fetch_fsmap learned FSMap version 32 2024-09-12T03:33:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3c06ea920 con 0x55a3c0665b00 2024-09-12T03:33:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.042+0000 7f841b400640 1 -- 172.21.15.50:0/3451667768 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84200043e0 con 0x55a3c0665b00 2024-09-12T03:33:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f841b400640 1 -- 172.21.15.50:0/3451667768 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8428003960 con 0x55a3c0665b00 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f843f027740 10 client.25750 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f843f027740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f843f027740 20 client.25750 populate_metadata read hostname 'smithi050' 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f843f027740 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c06eafe0 0x55a3c06ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f843f027740 4 client.25750 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3c05d2b80 con 0x55a3c06eafe0 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f8438e00640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c06eafe0 0x55a3c06ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f8438e00640 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c06eafe0 0x55a3c06ed400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939b86bfc0eb9674 server_cookie=42f1dc8b9dc66e17 in_seq=0 out_seq=0 2024-09-12T03:33:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.043+0000 7f841b400640 10 client.25750 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f841b400640 1 -- 172.21.15.50:0/3451667768 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3c05d2b80 con 0x55a3c06eafe0 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f841b400640 10 client.25750 handle_command_reply: tid=0 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 client.25750 shutdown 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 msgr2=0x55a3c0678b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c0678b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8428022a00 tx=0x7f8428022d90 comp rx=0 tx=0).stop 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 shutdown_connections 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3c06703a0 0x55a3c0669d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x55a3c06eafe0 0x55a3c06ed400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3c0665b00 0x55a3c0678b90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 --2- 172.21.15.50:0/3451667768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a3c06651e0 0x55a3c0678650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f841b400640 0 client.25750 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 >> 172.21.15.50:0/3451667768 conn(0x55a3c0661750 msgr2=0x55a3c066d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 shutdown_connections 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 1 -- 172.21.15.50:0/3451667768 wait complete. 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.061+0000 7f843f027740 20 client.25750 trim_cache size 0 max 16384 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:33:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f841c0a2f00 msgr2=0x7f841c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f841c0a2f00 0x7f841c0a53c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f842c0c4db0 tx=0x7f842c02a750 comp rx=0 tx=0).stop 2024-09-12T03:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 msgr2=0x7f843410b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f843410b290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f842c009b30 tx=0x7f842c0305d0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 shutdown_connections 2024-09-12T03:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f843410e3a0 0x7f843410b290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f841c0a2f00 0x7f841c0a53c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8434074ec0 0x7f843410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 --2- 172.21.15.50:0/1930127837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84340745a0 0x7f843410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 >> 172.21.15.50:0/1930127837 conn(0x7f84340fd7e0 msgr2=0x7f84340fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 shutdown_connections 2024-09-12T03:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:33:55.062+0000 7f843c200640 1 -- 172.21.15.50:0/1930127837 wait complete. 2024-09-12T03:33:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:56 smithi142 ceph-mon[35329]: pgmap v206: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1006 KiB/s wr, 345 op/s 2024-09-12T03:33:56.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:56 smithi082 ceph-mon[35046]: pgmap v206: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1006 KiB/s wr, 345 op/s 2024-09-12T03:33:56.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:56 smithi050 ceph-mon[30378]: pgmap v206: 129 pgs: 129 active+clean; 129 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1006 KiB/s wr, 345 op/s 2024-09-12T03:33:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:33:58 smithi142 ceph-mon[35329]: pgmap v207: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 453 op/s 2024-09-12T03:33:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:33:58 smithi082 ceph-mon[35046]: pgmap v207: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 453 op/s 2024-09-12T03:33:58.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:33:58 smithi050 ceph-mon[30378]: pgmap v207: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 453 op/s 2024-09-12T03:34:00.717 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:00 smithi050 ceph-mon[30378]: pgmap v208: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 946 KiB/s wr, 304 op/s 2024-09-12T03:34:00.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:00 smithi142 ceph-mon[35329]: pgmap v208: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 946 KiB/s wr, 304 op/s 2024-09-12T03:34:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:00 smithi082 ceph-mon[35046]: pgmap v208: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 946 KiB/s wr, 304 op/s 2024-09-12T03:34:01.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:01 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:01.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:01 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:01.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:01 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:02.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:02 smithi142 ceph-mon[35329]: pgmap v209: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 911 KiB/s wr, 291 op/s 2024-09-12T03:34:02.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:02 smithi082 ceph-mon[35046]: pgmap v209: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 911 KiB/s wr, 291 op/s 2024-09-12T03:34:02.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:02 smithi050 ceph-mon[30378]: pgmap v209: 129 pgs: 129 active+clean; 115 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 911 KiB/s wr, 291 op/s 2024-09-12T03:34:04.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:04 smithi142 ceph-mon[35329]: pgmap v210: 129 pgs: 129 active+clean; 125 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 583 op/s 2024-09-12T03:34:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:04 smithi082 ceph-mon[35046]: pgmap v210: 129 pgs: 129 active+clean; 125 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 583 op/s 2024-09-12T03:34:04.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:04 smithi050 ceph-mon[30378]: pgmap v210: 129 pgs: 129 active+clean; 125 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 583 op/s 2024-09-12T03:34:06.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:06 smithi142 ceph-mon[35329]: pgmap v211: 129 pgs: 129 active+clean; 125 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-12T03:34:06.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:06 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:06.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:06 smithi082 ceph-mon[35046]: pgmap v211: 129 pgs: 129 active+clean; 125 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-12T03:34:06.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:06 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:06 smithi050 ceph-mon[30378]: pgmap v211: 129 pgs: 129 active+clean; 125 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-12T03:34:06.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:06 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 -- 172.21.15.50:0/1918985115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb174104900 msgr2=0x7fb174108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1918985115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb174104900 0x7fb174108940 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a410 comp rx=0 tx=0).stop 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 -- 172.21.15.50:0/1918985115 shutdown_connections 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1918985115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1741090a0 0x7fb174110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1918985115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb174104900 0x7fb174108940 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1918985115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb174103f50 0x7fb174104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 -- 172.21.15.50:0/1918985115 >> 172.21.15.50:0/1918985115 conn(0x7fb1740ff9e0 msgr2=0x7fb174101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 -- 172.21.15.50:0/1918985115 shutdown_connections 2024-09-12T03:34:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.181+0000 7fb17bc00640 1 -- 172.21.15.50:0/1918985115 wait complete. 2024-09-12T03:34:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 Processor -- start 2024-09-12T03:34:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 -- start start 2024-09-12T03:34:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 0x7fb17419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1741090a0 0x7fb17419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 0x7fb17419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741140a0 con 0x7fb1741090a0 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113f20 con 0x7fb174103f50 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb17bc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb174114220 con 0x7fb17419ee70 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 0x7fb17419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb179200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 0x7fb17419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb173e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1741090a0 0x7fb17419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb179200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 0x7fb17419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:35454/0 (socket says 172.21.15.50:35454) 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 0x7fb17419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42906/0 (socket says 172.21.15.50:42906) 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.182+0000 7fb179200640 1 -- 172.21.15.50:0/1231401393 learned_addr learned my addr 172.21.15.50:0/1231401393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179200640 1 -- 172.21.15.50:0/1231401393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 msgr2=0x7fb17419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179200640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 0x7fb17419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179200640 1 -- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1741090a0 msgr2=0x7fb17419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179200640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1741090a0 0x7fb17419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179200640 1 -- 172.21.15.50:0/1231401393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17419f960 con 0x7fb174103f50 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179c00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 0x7fb17419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb179200640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 0x7fb17419a5b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e9f0 tx=0x7fb16800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.183+0000 7fb171600640 1 -- 172.21.15.50:0/1231401393 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16800ccc0 con 0x7fb174103f50 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.184+0000 7fb171600640 1 -- 172.21.15.50:0/1231401393 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb168010910 con 0x7fb174103f50 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.184+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17419fc50 con 0x7fb174103f50 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.184+0000 7fb171600640 1 -- 172.21.15.50:0/1231401393 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680188a0 con 0x7fb174103f50 2024-09-12T03:34:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.184+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741a7410 con 0x7fb174103f50 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 Processor -- start 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 -- start start 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb171600640 1 -- 172.21.15.50:0/1231401393 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb168010ac0 con 0x7fb174103f50 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x5621001395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100143e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56210009f100 con 0x5621001391e0 2024-09-12T03:34:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5621000471b0 con 0x562100139b00 2024-09-12T03:34:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb17ec40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56210009b0c0 con 0x5621001443a0 2024-09-12T03:34:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb179c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x5621001395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb179c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x5621001395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47770/0 (socket says 172.21.15.50:47770) 2024-09-12T03:34:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb173e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb179c00640 1 -- 172.21.15.50:0/3559380833 learned_addr learned my addr 172.21.15.50:0/3559380833 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:34:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.187+0000 7fb179200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb173e00640 1 -- 172.21.15.50:0/3559380833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 msgr2=0x562100143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb173e00640 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb173e00640 1 -- 172.21.15.50:0/3559380833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 msgr2=0x5621001395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb173e00640 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x5621001395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb173e00640 1 -- 172.21.15.50:0/3559380833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ffdb2210 con 0x5621001443a0 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb179c00640 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x5621001395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb173e00640 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100146790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d7fb6584164e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb171600640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb15c0a2da0 0x7fb15c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.188+0000 7fb166000640 1 -- 172.21.15.50:0/3559380833 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160040070 con 0x5621001443a0 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb166000640 1 -- 172.21.15.50:0/3559380833 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160005500 con 0x5621001443a0 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb166000640 1 -- 172.21.15.50:0/3559380833 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035040 con 0x5621001443a0 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb171600640 1 -- 172.21.15.50:0/1231401393 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fb1680cfc30 con 0x7fb174103f50 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb179c00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb15c0a2da0 0x7fb15c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 -- 172.21.15.50:0/3559380833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 msgr2=0x562100146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100146790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160005bd0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 -- 172.21.15.50:0/3559380833 shutdown_connections 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100146790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 --2- 172.21.15.50:0/3559380833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x5621001395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb179c00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb15c0a2da0 0x7fb15c0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb16c012bb0 tx=0x7fb16c009500 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 -- 172.21.15.50:0/3559380833 >> 172.21.15.50:0/3559380833 conn(0x562100135750 msgr2=0x562100135b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 -- 172.21.15.50:0/3559380833 shutdown_connections 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.189+0000 7fb17ec40740 1 -- 172.21.15.50:0/3559380833 wait complete. 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.190+0000 7fb17ec40740 1 -- start start 2024-09-12T03:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.190+0000 7fb17ec40740 1 Processor -- start 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001391e0 0x5621001413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x5621001418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100141e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562100146f70 con 0x562100139b00 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562100028090 con 0x5621001391e0 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5621001470f0 con 0x5621001443a0 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb17ec40740 10 client.0 _ll_register_callbacks cb 0x56210007cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb173e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x5621001418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42916/0 (socket says 172.21.15.50:42916) 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb173e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x5621001418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47786/0 (socket says 172.21.15.50:47786) 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 -- 172.21.15.50:0/2218560521 learned_addr learned my addr 172.21.15.50:0/2218560521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179200640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001391e0 0x5621001413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 -- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001391e0 msgr2=0x5621001413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001391e0 0x5621001413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 -- 172.21.15.50:0/2218560521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 msgr2=0x5621001418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x5621001418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb179c00640 1 -- 172.21.15.50:0/2218560521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ffe7e620 con 0x5621001443a0 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.191+0000 7fb173e00640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x5621001418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb179c00640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100141e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb16c012190 tx=0x7fb16c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefaa54f970edc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb15aa00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb15aa00640 1 -- 172.21.15.50:0/2218560521 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c011d50 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160005500 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb15aa00640 1 -- 172.21.15.50:0/2218560521 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fb16c003970 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb15aa00640 1 -- 172.21.15.50:0/2218560521 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb16c032050 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb17ec40740 10 client.25762 fetch_fsmap learned FSMap version 34 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.192+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562100152cd0 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb15aa00640 1 -- 172.21.15.50:0/2218560521 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160005500 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb15aa00640 1 -- 172.21.15.50:0/2218560521 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fb16c027020 con 0x5621001443a0 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb17ec40740 10 client.25762 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb17ec40740 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb17ec40740 20 client.25762 populate_metadata read hostname 'smithi050' 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x562100154140 0x562100156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb17ec40740 4 client.25762 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:34:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621000ac830 con 0x562100154140 2024-09-12T03:34:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.193+0000 7fb179200640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x562100154140 0x562100156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.194+0000 7fb179200640 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x562100154140 0x562100156530 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fe42273f5bcfd5 server_cookie=6aeefb88b3cd44e3 in_seq=0 out_seq=0 2024-09-12T03:34:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.194+0000 7fb15aa00640 10 client.25762 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb15aa00640 1 -- 172.21.15.50:0/2218560521 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621000ac830 con 0x562100154140 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb15aa00640 10 client.25762 handle_command_reply: tid=0 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 client.25762 shutdown 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 msgr2=0x562100141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100141e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb16c012190 tx=0x7fb16c004ab0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 shutdown_connections 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x562100154140 0x562100156530 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100141e30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x5621001418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2218560521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001391e0 0x5621001413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb15aa00640 0 client.25762 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:34:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.195+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 >> 172.21.15.50:0/2218560521 conn(0x562100135750 msgr2=0x56210013b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.196+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 shutdown_connections 2024-09-12T03:34:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.196+0000 7fb17ec40740 1 -- 172.21.15.50:0/2218560521 wait complete. 2024-09-12T03:34:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.196+0000 7fb17ec40740 20 client.25762 trim_cache size 0 max 16384 2024-09-12T03:34:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.211+0000 7fb17ec40740 1 Processor -- start 2024-09-12T03:34:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 -- start start 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 0x56210013b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x56210015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001443a0 0x56210015c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741b3a20 con 0x562100139b00 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741b12f0 con 0x5621001443a0 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb17ec40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741af140 con 0x5621001391e0 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 0x56210013b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb173e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x56210015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 0x56210013b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42932/0 (socket says 172.21.15.50:42932) 2024-09-12T03:34:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 -- 172.21.15.50:0/2126327776 learned_addr learned my addr 172.21.15.50:0/2126327776 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179c00640 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001443a0 0x56210015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 -- 172.21.15.50:0/2126327776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001443a0 msgr2=0x56210015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001443a0 0x56210015c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 -- 172.21.15.50:0/2126327776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 msgr2=0x56210015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x56210015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.212+0000 7fb179200640 1 -- 172.21.15.50:0/2126327776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562100130d60 con 0x5621001391e0 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.213+0000 7fb179200640 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 0x56210013b3a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb168010f50 tx=0x7fb1680d36e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30772c45fe1b533f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.213+0000 7fb165600640 1 -- 172.21.15.50:0/2126327776 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16809b8a0 con 0x5621001391e0 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.213+0000 7fb165600640 1 -- 172.21.15.50:0/2126327776 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16809c460 con 0x5621001391e0 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.213+0000 7fb165600640 1 -- 172.21.15.50:0/2126327776 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16809b6c0 con 0x5621001391e0 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 -- 172.21.15.50:0/2126327776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 msgr2=0x56210013b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 0x56210013b3a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb168010f50 tx=0x7fb1680d36e0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 -- 172.21.15.50:0/2126327776 shutdown_connections 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5621001443a0 0x56210015c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562100139b00 0x56210015be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2126327776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001391e0 0x56210013b3a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 -- 172.21.15.50:0/2126327776 >> 172.21.15.50:0/2126327776 conn(0x562100135750 msgr2=0x56210013ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 -- 172.21.15.50:0/2126327776 shutdown_connections 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 -- 172.21.15.50:0/2126327776 wait complete. 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 -- start start 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.214+0000 7fb17ec40740 1 Processor -- start 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x56210014c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100147b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100148070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741af7b0 con 0x5621001391e0 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741b0c20 con 0x562100139b00 2024-09-12T03:34:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741afe80 con 0x5621001443a0 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb17ec40740 10 client.0 _ll_register_callbacks cb 0x7fb16c0128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb173e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb179200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x56210014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:42946/0 (socket says 172.21.15.50:42946) 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb173e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:35496/0 (socket says 172.21.15.50:35496) 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb179c00640 1 -- 172.21.15.50:0/2072471835 learned_addr learned my addr 172.21.15.50:0/2072471835 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb179c00640 1 -- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 msgr2=0x562100147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.215+0000 7fb179c00640 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb179c00640 1 -- 172.21.15.50:0/2072471835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 msgr2=0x56210014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb179c00640 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x56210014c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb179c00640 1 -- 172.21.15.50:0/2072471835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562100148630 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb173e00640 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb179c00640 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100148070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0294e0 tx=0x7fb16c029810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81fd34562525c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb15aa00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb15aa00640 1 -- 172.21.15.50:0/2072471835 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c03a020 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16809c460 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb15aa00640 1 -- 172.21.15.50:0/2072471835 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9437+0+0 (secure 0 0 0) 0x7fb16c029930 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb15aa00640 1 -- 172.21.15.50:0/2072471835 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb16c045050 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb17ec40740 10 client.25768 fetch_fsmap learned FSMap version 34 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.216+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621001beb70 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb15aa00640 1 -- 172.21.15.50:0/2072471835 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16809c460 con 0x5621001443a0 2024-09-12T03:34:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb15aa00640 1 -- 172.21.15.50:0/2072471835 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fb16c027050 con 0x5621001443a0 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb17ec40740 10 client.25768 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb17ec40740 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb17ec40740 20 client.25768 populate_metadata read hostname 'smithi050' 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5621001bf250 0x5621001c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb17ec40740 4 client.25768 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 --> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621000a6b80 con 0x5621001bf250 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.217+0000 7fb179200640 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5621001bf250 0x5621001c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.218+0000 7fb179200640 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5621001bf250 0x5621001c1670 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d36382eb06caa34 server_cookie=76be4b8a33ab7795 in_seq=0 out_seq=0 2024-09-12T03:34:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.218+0000 7fb15aa00640 10 client.25768 ms_handle_connect on v2:172.21.15.82:6838/2348373979 2024-09-12T03:34:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb15aa00640 1 -- 172.21.15.50:0/2072471835 <== mds.0 v2:172.21.15.82:6838/2348373979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621000a6b80 con 0x5621001bf250 2024-09-12T03:34:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb15aa00640 10 client.25768 handle_command_reply: tid=0 2024-09-12T03:34:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 client.25768 shutdown 2024-09-12T03:34:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 msgr2=0x562100148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100148070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0294e0 tx=0x7fb16c029810 comp rx=0 tx=0).stop 2024-09-12T03:34:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 shutdown_connections 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:6838/2348373979,v1:172.21.15.82:6839/2348373979] conn(0x5621001bf250 0x5621001c1670 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621001443a0 0x562100148070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562100139b00 0x562100147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb15aa00640 0 client.25768 ms_handle_reset on v2:172.21.15.82:6838/2348373979 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 --2- 172.21.15.50:0/2072471835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621001391e0 0x56210014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 >> 172.21.15.50:0/2072471835 conn(0x562100135750 msgr2=0x56210013f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 shutdown_connections 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 1 -- 172.21.15.50:0/2072471835 wait complete. 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.240+0000 7fb17ec40740 20 client.25768 trim_cache size 0 max 16384 2024-09-12T03:34:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-09-12T03:34:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "message": "", 2024-09-12T03:34:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-09-12T03:34:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-09-12T03:34:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb15c0a2da0 msgr2=0x7fb15c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb15c0a2da0 0x7fb15c0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb16c012bb0 tx=0x7fb16c009500 comp rx=0 tx=0).stop 2024-09-12T03:34:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 msgr2=0x7fb17419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 0x7fb17419a5b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e9f0 tx=0x7fb16800edf0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 shutdown_connections 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb17419ee70 0x7fb17419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb15c0a2da0 0x7fb15c0a5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1741090a0 0x7fb17419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 --2- 172.21.15.50:0/1231401393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174103f50 0x7fb17419a5b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 >> 172.21.15.50:0/1231401393 conn(0x7fb1740ff9e0 msgr2=0x7fb174100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 shutdown_connections 2024-09-12T03:34:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-09-12T03:34:08.241+0000 7fb17bc00640 1 -- 172.21.15.50:0/1231401393 wait complete. 2024-09-12T03:34:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:08 smithi142 ceph-mon[35329]: pgmap v212: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 444 op/s 2024-09-12T03:34:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:08 smithi082 ceph-mon[35046]: pgmap v212: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 444 op/s 2024-09-12T03:34:08.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:08 smithi050 ceph-mon[30378]: pgmap v212: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 444 op/s 2024-09-12T03:34:10.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:10 smithi142 ceph-mon[35329]: pgmap v213: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.0 MiB/s wr, 337 op/s 2024-09-12T03:34:10.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:10 smithi082 ceph-mon[35046]: pgmap v213: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.0 MiB/s wr, 337 op/s 2024-09-12T03:34:10.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:10 smithi050 ceph-mon[30378]: pgmap v213: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.0 MiB/s wr, 337 op/s 2024-09-12T03:34:11.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:11 smithi142 ceph-mon[35329]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:34:11.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:11 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:11 smithi082 ceph-mon[35046]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:34:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:11 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:11.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:11 smithi050 ceph-mon[30378]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:34:11.915 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:11 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:12 smithi142 ceph-mon[35329]: pgmap v215: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 394 op/s 2024-09-12T03:34:12.814 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:12 smithi050 ceph-mon[30378]: pgmap v215: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 394 op/s 2024-09-12T03:34:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:12 smithi082 ceph-mon[35046]: pgmap v215: 129 pgs: 129 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 394 op/s 2024-09-12T03:34:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:14 smithi142 ceph-mon[35329]: pgmap v216: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 136 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-12T03:34:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:14 smithi082 ceph-mon[35046]: pgmap v216: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 136 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-12T03:34:14.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:14 smithi050 ceph-mon[30378]: pgmap v216: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 136 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-12T03:34:15.038 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-09-12T03:34:15.038 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:34:16.316 DEBUG:teuthology.parallel:result is None 2024-09-12T03:34:16.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:34:16.380 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:34:16.380 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T03:34:16.391 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:34:16.403 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T03:34:16.414 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T03:34:16.414 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:34:16.414 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:34:16.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T03:34:16.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T03:34:16.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:16 smithi050 ceph-mon[30378]: pgmap v217: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 136 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-12T03:34:16.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:16 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:16 smithi082 ceph-mon[35046]: pgmap v217: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 136 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-12T03:34:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:16 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:16 smithi142 ceph-mon[35329]: pgmap v217: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 136 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-12T03:34:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:16 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:34:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:18 smithi082 ceph-mon[35046]: pgmap v218: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 373 op/s 2024-09-12T03:34:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:18 smithi050 ceph-mon[30378]: pgmap v218: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 373 op/s 2024-09-12T03:34:19.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:18 smithi142 ceph-mon[35329]: pgmap v218: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 373 op/s 2024-09-12T03:34:19.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:19 smithi050 ceph-mon[30378]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:34:20.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:19 smithi142 ceph-mon[35329]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:34:20.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:19 smithi082 ceph-mon[35046]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:34:20.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:20 smithi050 ceph-mon[30378]: pgmap v220: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 436 op/s 2024-09-12T03:34:21.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:20 smithi142 ceph-mon[35329]: pgmap v220: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 436 op/s 2024-09-12T03:34:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:20 smithi082 ceph-mon[35046]: pgmap v220: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 436 op/s 2024-09-12T03:34:22.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:22 smithi050 ceph-mon[30378]: pgmap v221: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 383 op/s 2024-09-12T03:34:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:22 smithi142 ceph-mon[35329]: pgmap v221: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 383 op/s 2024-09-12T03:34:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:22 smithi082 ceph-mon[35046]: pgmap v221: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 383 op/s 2024-09-12T03:34:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:24 smithi050 ceph-mon[30378]: pgmap v222: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 668 KiB/s wr, 177 op/s 2024-09-12T03:34:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:24 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:34:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:24 smithi142 ceph-mon[35329]: pgmap v222: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 668 KiB/s wr, 177 op/s 2024-09-12T03:34:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:24 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:34:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:24 smithi082 ceph-mon[35046]: pgmap v222: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 668 KiB/s wr, 177 op/s 2024-09-12T03:34:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:24 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:34:25.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:25 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:34:25.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:25 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:34:25.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:25 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:34:26.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:25 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:34:26.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:25 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:34:26.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:25 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:34:26.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:25 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:34:26.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:25 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:34:26.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:25 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:34:27.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:26 smithi142 ceph-mon[35329]: pgmap v223: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 668 KiB/s wr, 177 op/s 2024-09-12T03:34:27.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:26 smithi082 ceph-mon[35046]: pgmap v223: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 668 KiB/s wr, 177 op/s 2024-09-12T03:34:27.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:26 smithi050 ceph-mon[30378]: pgmap v223: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 668 KiB/s wr, 177 op/s 2024-09-12T03:34:29.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:28 smithi142 ceph-mon[35329]: pgmap v224: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-09-12T03:34:29.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:28 smithi082 ceph-mon[35046]: pgmap v224: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-09-12T03:34:29.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:28 smithi050 ceph-mon[30378]: pgmap v224: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 23 op/s 2024-09-12T03:34:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:30 smithi142 ceph-mon[35329]: pgmap v225: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T03:34:31.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:30 smithi082 ceph-mon[35046]: pgmap v225: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T03:34:31.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:30 smithi050 ceph-mon[30378]: pgmap v225: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T03:34:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:33 smithi142 ceph-mon[35329]: pgmap v226: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-09-12T03:34:33.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:33 smithi082 ceph-mon[35046]: pgmap v226: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-09-12T03:34:33.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:33 smithi050 ceph-mon[30378]: pgmap v226: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-09-12T03:34:35.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:35 smithi142 ceph-mon[35329]: pgmap v227: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T03:34:35.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:35 smithi082 ceph-mon[35046]: pgmap v227: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T03:34:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:35 smithi050 ceph-mon[30378]: pgmap v227: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T03:34:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:37 smithi142 ceph-mon[35329]: pgmap v228: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:34:37.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:37 smithi082 ceph-mon[35046]: pgmap v228: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:34:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:37 smithi050 ceph-mon[30378]: pgmap v228: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:34:39.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:39 smithi142 ceph-mon[35329]: pgmap v229: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T03:34:39.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:39 smithi082 ceph-mon[35046]: pgmap v229: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T03:34:39.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:39 smithi050 ceph-mon[30378]: pgmap v229: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T03:34:41.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:41 smithi142 ceph-mon[35329]: pgmap v230: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-12T03:34:41.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:41 smithi082 ceph-mon[35046]: pgmap v230: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-12T03:34:41.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:41 smithi050 ceph-mon[30378]: pgmap v230: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-12T03:34:43.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:43 smithi142 ceph-mon[35329]: pgmap v231: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-12T03:34:43.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:43 smithi082 ceph-mon[35046]: pgmap v231: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-12T03:34:43.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:43 smithi050 ceph-mon[30378]: pgmap v231: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-12T03:34:45.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:45 smithi142 ceph-mon[35329]: pgmap v232: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T03:34:45.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:45 smithi082 ceph-mon[35046]: pgmap v232: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T03:34:45.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:45 smithi050 ceph-mon[30378]: pgmap v232: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T03:34:47.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:47 smithi142 ceph-mon[35329]: pgmap v233: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:47.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:47 smithi082 ceph-mon[35046]: pgmap v233: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:47.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:47 smithi050 ceph-mon[30378]: pgmap v233: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:49.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:49 smithi142 ceph-mon[35329]: pgmap v234: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:34:49.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:49 smithi082 ceph-mon[35046]: pgmap v234: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:34:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:49 smithi050 ceph-mon[30378]: pgmap v234: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:34:51.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:51 smithi142 ceph-mon[35329]: pgmap v235: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:51.698 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:51 smithi050 ceph-mon[30378]: pgmap v235: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:51.698 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:51 smithi082 ceph-mon[35046]: pgmap v235: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:53.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:53 smithi142 ceph-mon[35329]: pgmap v236: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:53 smithi082 ceph-mon[35046]: pgmap v236: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:53.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:53 smithi050 ceph-mon[30378]: pgmap v236: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:34:54.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:54 smithi142 ceph-mon[35329]: pgmap v237: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:34:54.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:54 smithi082 ceph-mon[35046]: pgmap v237: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:34:54.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:54 smithi050 ceph-mon[30378]: pgmap v237: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:34:56.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:56 smithi050 ceph-mon[30378]: pgmap v238: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:34:56.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:56 smithi142 ceph-mon[35329]: pgmap v238: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:34:56.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:56 smithi082 ceph-mon[35046]: pgmap v238: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:34:58.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:34:58 smithi050 ceph-mon[30378]: pgmap v239: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:34:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:34:58 smithi142 ceph-mon[35329]: pgmap v239: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:34:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:34:58 smithi082 ceph-mon[35046]: pgmap v239: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:00.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:00 smithi050 ceph-mon[30378]: pgmap v240: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:00.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:00 smithi142 ceph-mon[35329]: pgmap v240: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:00.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:00 smithi082 ceph-mon[35046]: pgmap v240: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:02.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:02 smithi050 ceph-mon[30378]: pgmap v241: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:02.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:02 smithi142 ceph-mon[35329]: pgmap v241: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:02.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:02 smithi082 ceph-mon[35046]: pgmap v241: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:04.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:04 smithi050 ceph-mon[30378]: pgmap v242: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:35:04.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:04 smithi142 ceph-mon[35329]: pgmap v242: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:35:04.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:04 smithi082 ceph-mon[35046]: pgmap v242: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:35:06.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:06 smithi050 ceph-mon[30378]: pgmap v243: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:35:06.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:06 smithi142 ceph-mon[35329]: pgmap v243: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:35:06.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:06 smithi082 ceph-mon[35046]: pgmap v243: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:35:08.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:08 smithi050 ceph-mon[30378]: pgmap v244: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:35:08.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:08 smithi142 ceph-mon[35329]: pgmap v244: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:35:08.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:08 smithi082 ceph-mon[35046]: pgmap v244: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:35:10.656 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:10 smithi082 ceph-mon[35046]: pgmap v245: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:35:10.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:10 smithi050 ceph-mon[30378]: pgmap v245: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:35:10.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:10 smithi142 ceph-mon[35329]: pgmap v245: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:35:11.665 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:11 smithi050 ceph-mon[30378]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:35:11.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:11 smithi142 ceph-mon[35329]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:35:11.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:11 smithi082 ceph-mon[35046]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:35:12.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:12 smithi050 ceph-mon[30378]: pgmap v247: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:35:12.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:12 smithi142 ceph-mon[35329]: pgmap v247: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:35:12.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:12 smithi082 ceph-mon[35046]: pgmap v247: 129 pgs: 129 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:35:14.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:14 smithi050 ceph-mon[30378]: pgmap v248: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:14.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:14 smithi142 ceph-mon[35329]: pgmap v248: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:14.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:14 smithi082 ceph-mon[35046]: pgmap v248: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:16.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:16 smithi050 ceph-mon[30378]: pgmap v249: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:16.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:16 smithi142 ceph-mon[35329]: pgmap v249: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:16.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:16 smithi082 ceph-mon[35046]: pgmap v249: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 142 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:18.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:18 smithi142 ceph-mon[35329]: pgmap v250: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:18.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:18 smithi082 ceph-mon[35046]: pgmap v250: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:18.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:18 smithi050 ceph-mon[30378]: pgmap v250: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:20.717 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:20 smithi050 ceph-mon[30378]: pgmap v251: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:20.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:20 smithi142 ceph-mon[35329]: pgmap v251: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:20.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:20 smithi082 ceph-mon[35046]: pgmap v251: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:35:22.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:22 smithi142 ceph-mon[35329]: pgmap v252: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 5 op/s 2024-09-12T03:35:22.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:22 smithi082 ceph-mon[35046]: pgmap v252: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 5 op/s 2024-09-12T03:35:22.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:22 smithi050 ceph-mon[30378]: pgmap v252: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 5 op/s 2024-09-12T03:35:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:24 smithi050 ceph-mon[30378]: pgmap v253: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T03:35:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:24 smithi050 ceph-mon[30378]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:35:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:24 smithi142 ceph-mon[35329]: pgmap v253: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T03:35:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:24 smithi142 ceph-mon[35329]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:35:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:24 smithi082 ceph-mon[35046]: pgmap v253: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T03:35:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:24 smithi082 ceph-mon[35046]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:35:26.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:25 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:35:26.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:25 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:35:26.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:25 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:35:27.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:26 smithi142 ceph-mon[35329]: pgmap v255: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:35:27.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:26 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:35:27.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:26 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:35:27.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:26 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:35:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:26 smithi082 ceph-mon[35046]: pgmap v255: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:35:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:26 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:35:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:26 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:35:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:26 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:35:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:26 smithi050 ceph-mon[30378]: pgmap v255: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:35:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:26 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:35:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:26 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:35:27.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:26 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:35:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:28 smithi142 ceph-mon[35329]: pgmap v256: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:29.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:28 smithi082 ceph-mon[35046]: pgmap v256: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:28 smithi050 ceph-mon[30378]: pgmap v256: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:31.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:30 smithi142 ceph-mon[35329]: pgmap v257: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:30 smithi082 ceph-mon[35046]: pgmap v257: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:30 smithi050 ceph-mon[30378]: pgmap v257: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:33.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:32 smithi142 ceph-mon[35329]: pgmap v258: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:32 smithi082 ceph-mon[35046]: pgmap v258: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:32 smithi050 ceph-mon[30378]: pgmap v258: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:35.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:34 smithi142 ceph-mon[35329]: pgmap v259: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:34 smithi082 ceph-mon[35046]: pgmap v259: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:34 smithi050 ceph-mon[30378]: pgmap v259: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:35:37.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:36 smithi142 ceph-mon[35329]: pgmap v260: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:35:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:36 smithi082 ceph-mon[35046]: pgmap v260: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:35:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:36 smithi050 ceph-mon[30378]: pgmap v260: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:35:39.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:38 smithi142 ceph-mon[35329]: pgmap v261: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:38 smithi082 ceph-mon[35046]: pgmap v261: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:38 smithi050 ceph-mon[30378]: pgmap v261: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:41.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:40 smithi142 ceph-mon[35329]: pgmap v262: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:40 smithi082 ceph-mon[35046]: pgmap v262: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:40 smithi050 ceph-mon[30378]: pgmap v262: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:43.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:42 smithi142 ceph-mon[35329]: pgmap v263: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:42 smithi082 ceph-mon[35046]: pgmap v263: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:42 smithi050 ceph-mon[30378]: pgmap v263: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:45.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:44 smithi142 ceph-mon[35329]: pgmap v264: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:44 smithi082 ceph-mon[35046]: pgmap v264: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:44 smithi050 ceph-mon[30378]: pgmap v264: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:47.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:46 smithi142 ceph-mon[35329]: pgmap v265: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:46 smithi082 ceph-mon[35046]: pgmap v265: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:47.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:46 smithi050 ceph-mon[30378]: pgmap v265: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:49.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:48 smithi142 ceph-mon[35329]: pgmap v266: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:48 smithi082 ceph-mon[35046]: pgmap v266: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:48 smithi050 ceph-mon[30378]: pgmap v266: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:50.966 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:50 smithi050 ceph-mon[30378]: pgmap v267: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:51.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:50 smithi142 ceph-mon[35329]: pgmap v267: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:50 smithi082 ceph-mon[35046]: pgmap v267: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:53.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:52 smithi142 ceph-mon[35329]: pgmap v268: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:53.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:52 smithi082 ceph-mon[35046]: pgmap v268: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:52 smithi050 ceph-mon[30378]: pgmap v268: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:55.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:54 smithi142 ceph-mon[35329]: pgmap v269: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:55.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:54 smithi082 ceph-mon[35046]: pgmap v269: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:55.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:54 smithi050 ceph-mon[30378]: pgmap v269: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:57.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:56 smithi142 ceph-mon[35329]: pgmap v270: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:57.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:56 smithi082 ceph-mon[35046]: pgmap v270: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:56 smithi050 ceph-mon[30378]: pgmap v270: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:35:59.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:35:58 smithi142 ceph-mon[35329]: pgmap v271: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:59.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:35:58 smithi082 ceph-mon[35046]: pgmap v271: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:35:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:35:58 smithi050 ceph-mon[30378]: pgmap v271: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:01.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:00 smithi142 ceph-mon[35329]: pgmap v272: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:01.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:00 smithi082 ceph-mon[35046]: pgmap v272: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:00 smithi050 ceph-mon[30378]: pgmap v272: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:03.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:02 smithi142 ceph-mon[35329]: pgmap v273: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:03.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:02 smithi082 ceph-mon[35046]: pgmap v273: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:02 smithi050 ceph-mon[30378]: pgmap v273: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:05.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:04 smithi142 ceph-mon[35329]: pgmap v274: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:36:05.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:04 smithi082 ceph-mon[35046]: pgmap v274: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:36:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:04 smithi050 ceph-mon[30378]: pgmap v274: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:36:07.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:06 smithi142 ceph-mon[35329]: pgmap v275: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:36:07.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:06 smithi082 ceph-mon[35046]: pgmap v275: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:36:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:06 smithi050 ceph-mon[30378]: pgmap v275: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:36:09.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:08 smithi142 ceph-mon[35329]: pgmap v276: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:36:09.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:08 smithi082 ceph-mon[35046]: pgmap v276: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:36:09.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:08 smithi050 ceph-mon[30378]: pgmap v276: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:36:11.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:10 smithi142 ceph-mon[35329]: pgmap v277: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:36:11.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:10 smithi082 ceph-mon[35046]: pgmap v277: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:36:11.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:10 smithi050 ceph-mon[30378]: pgmap v277: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:36:12.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:11 smithi142 ceph-mon[35329]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:36:12.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:11 smithi082 ceph-mon[35046]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:36:12.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:11 smithi050 ceph-mon[30378]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:36:13.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:12 smithi142 ceph-mon[35329]: pgmap v279: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:36:13.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:12 smithi082 ceph-mon[35046]: pgmap v279: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:36:13.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:12 smithi050 ceph-mon[30378]: pgmap v279: 129 pgs: 129 active+clean; 142 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:36:15.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:14 smithi142 ceph-mon[35329]: pgmap v280: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:15.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:14 smithi082 ceph-mon[35046]: pgmap v280: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:15.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:14 smithi050 ceph-mon[30378]: pgmap v280: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:17.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:16 smithi142 ceph-mon[35329]: pgmap v281: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:17.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:16 smithi082 ceph-mon[35046]: pgmap v281: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:17.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:16 smithi050 ceph-mon[30378]: pgmap v281: 129 pgs: 20 active+clean+snaptrim, 59 active+clean+snaptrim_wait, 50 active+clean; 142 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:19.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:18 smithi142 ceph-mon[35329]: pgmap v282: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:19.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:18 smithi082 ceph-mon[35046]: pgmap v282: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:19.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:18 smithi050 ceph-mon[30378]: pgmap v282: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 141 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:21.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:20 smithi142 ceph-mon[35329]: pgmap v283: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:21.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:20 smithi082 ceph-mon[35046]: pgmap v283: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:20 smithi050 ceph-mon[30378]: pgmap v283: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:36:23.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:22 smithi142 ceph-mon[35329]: pgmap v284: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 193 B/s wr, 6 op/s 2024-09-12T03:36:23.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:22 smithi082 ceph-mon[35046]: pgmap v284: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 193 B/s wr, 6 op/s 2024-09-12T03:36:23.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:22 smithi050 ceph-mon[30378]: pgmap v284: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 193 B/s wr, 6 op/s 2024-09-12T03:36:25.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:24 smithi142 ceph-mon[35329]: pgmap v285: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T03:36:25.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:24 smithi142 ceph-mon[35329]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:36:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:24 smithi082 ceph-mon[35046]: pgmap v285: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T03:36:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:24 smithi082 ceph-mon[35046]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:36:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:24 smithi050 ceph-mon[30378]: pgmap v285: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-12T03:36:25.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:24 smithi050 ceph-mon[30378]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:36:27.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:26 smithi142 ceph-mon[35329]: pgmap v287: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:36:27.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:26 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:36:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:26 smithi082 ceph-mon[35046]: pgmap v287: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:36:27.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:26 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:36:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:26 smithi050 ceph-mon[30378]: pgmap v287: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:36:27.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:26 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:36:28.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:27 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:36:28.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:27 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:36:28.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:27 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:36:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:27 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:36:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:27 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:36:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:27 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:36:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:27 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:36:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:27 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:36:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:27 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:36:29.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:28 smithi082 ceph-mon[35046]: pgmap v288: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:28 smithi050 ceph-mon[30378]: pgmap v288: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:29.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:28 smithi142 ceph-mon[35329]: pgmap v288: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:31.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:30 smithi082 ceph-mon[35046]: pgmap v289: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:30 smithi050 ceph-mon[30378]: pgmap v289: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:30 smithi142 ceph-mon[35329]: pgmap v289: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:33.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:32 smithi082 ceph-mon[35046]: pgmap v290: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:33.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:32 smithi050 ceph-mon[30378]: pgmap v290: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:32 smithi142 ceph-mon[35329]: pgmap v290: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:35.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:34 smithi082 ceph-mon[35046]: pgmap v291: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:35.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:34 smithi050 ceph-mon[30378]: pgmap v291: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:35.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:34 smithi142 ceph-mon[35329]: pgmap v291: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:36:37.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:36 smithi082 ceph-mon[35046]: pgmap v292: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:36:37.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:36 smithi050 ceph-mon[30378]: pgmap v292: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:36:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:36 smithi142 ceph-mon[35329]: pgmap v292: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:36:39.096 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:38 smithi142 ceph-mon[35329]: pgmap v293: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:39.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:38 smithi082 ceph-mon[35046]: pgmap v293: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:39.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:38 smithi050 ceph-mon[30378]: pgmap v293: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:41.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:40 smithi082 ceph-mon[35046]: pgmap v294: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:41.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:40 smithi050 ceph-mon[30378]: pgmap v294: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:41.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:40 smithi142 ceph-mon[35329]: pgmap v294: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:43.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:42 smithi082 ceph-mon[35046]: pgmap v295: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:43.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:42 smithi050 ceph-mon[30378]: pgmap v295: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:43.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:42 smithi142 ceph-mon[35329]: pgmap v295: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:45.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:44 smithi082 ceph-mon[35046]: pgmap v296: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:45.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:44 smithi050 ceph-mon[30378]: pgmap v296: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:45.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:44 smithi142 ceph-mon[35329]: pgmap v296: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:47.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:46 smithi082 ceph-mon[35046]: pgmap v297: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:47.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:46 smithi050 ceph-mon[30378]: pgmap v297: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:47.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:46 smithi142 ceph-mon[35329]: pgmap v297: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:49.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:48 smithi082 ceph-mon[35046]: pgmap v298: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:49.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:48 smithi050 ceph-mon[30378]: pgmap v298: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:49.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:48 smithi142 ceph-mon[35329]: pgmap v298: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:51.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:50 smithi082 ceph-mon[35046]: pgmap v299: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:51.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:50 smithi050 ceph-mon[30378]: pgmap v299: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:51.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:50 smithi142 ceph-mon[35329]: pgmap v299: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:53.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:52 smithi050 ceph-mon[30378]: pgmap v300: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:53.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:52 smithi142 ceph-mon[35329]: pgmap v300: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:53.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:52 smithi082 ceph-mon[35046]: pgmap v300: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:55.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:54 smithi050 ceph-mon[30378]: pgmap v301: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:55.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:54 smithi142 ceph-mon[35329]: pgmap v301: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:55.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:54 smithi082 ceph-mon[35046]: pgmap v301: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:56 smithi050 ceph-mon[30378]: pgmap v302: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:57.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:56 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:36:57.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:56 smithi142 ceph-mon[35329]: pgmap v302: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:57.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:56 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:36:57.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:56 smithi082 ceph-mon[35046]: pgmap v302: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:36:57.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:56 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:36:59.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:36:58 smithi050 ceph-mon[30378]: pgmap v303: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:59.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:36:58 smithi142 ceph-mon[35329]: pgmap v303: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:36:59.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:36:58 smithi082 ceph-mon[35046]: pgmap v303: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:37:01.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:00 smithi050 ceph-mon[30378]: pgmap v304: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:01.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:00 smithi142 ceph-mon[35329]: pgmap v304: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:01.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:00 smithi082 ceph-mon[35046]: pgmap v304: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:03.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:02 smithi050 ceph-mon[30378]: pgmap v305: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:03.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:02 smithi142 ceph-mon[35329]: pgmap v305: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:03.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:02 smithi082 ceph-mon[35046]: pgmap v305: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:05.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:04 smithi050 ceph-mon[30378]: pgmap v306: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:37:05.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:04 smithi142 ceph-mon[35329]: pgmap v306: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:37:05.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:04 smithi082 ceph-mon[35046]: pgmap v306: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:37:07.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:06 smithi050 ceph-mon[30378]: pgmap v307: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:37:07.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:06 smithi142 ceph-mon[35329]: pgmap v307: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:37:07.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:06 smithi082 ceph-mon[35046]: pgmap v307: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:37:09.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:08 smithi050 ceph-mon[30378]: pgmap v308: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:37:09.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:08 smithi142 ceph-mon[35329]: pgmap v308: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:37:09.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:08 smithi082 ceph-mon[35046]: pgmap v308: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-12T03:37:11.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:10 smithi050 ceph-mon[30378]: pgmap v309: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:37:11.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:10 smithi142 ceph-mon[35329]: pgmap v309: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:37:11.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:10 smithi082 ceph-mon[35046]: pgmap v309: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-12T03:37:12.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:11 smithi050 ceph-mon[30378]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:37:12.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:11 smithi142 ceph-mon[35329]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:37:12.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:11 smithi082 ceph-mon[35046]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:37:13.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:12 smithi142 ceph-mon[35329]: pgmap v311: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:37:13.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:12 smithi082 ceph-mon[35046]: pgmap v311: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:37:13.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:12 smithi050 ceph-mon[30378]: pgmap v311: 129 pgs: 129 active+clean; 141 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-12T03:37:14.570 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:37:14.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.836+0000 7f1009200640 1 -- 172.21.15.50:0/1525497721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10041006a0 msgr2=0x7f1004100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.836+0000 7f1009200640 1 --2- 172.21.15.50:0/1525497721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10041006a0 0x7f1004100a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009ca0 tx=0x7f0ff402a8b0 comp rx=0 tx=0).stop 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.836+0000 7f1009200640 1 -- 172.21.15.50:0/1525497721 shutdown_connections 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.836+0000 7f1009200640 1 --2- 172.21.15.50:0/1525497721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f1004110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.836+0000 7f1009200640 1 --2- 172.21.15.50:0/1525497721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1004101050 0x7f100410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.836+0000 7f1009200640 1 --2- 172.21.15.50:0/1525497721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10041006a0 0x7f1004100a80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.837+0000 7f1009200640 1 -- 172.21.15.50:0/1525497721 >> 172.21.15.50:0/1525497721 conn(0x7f10040777c0 msgr2=0x7f10040feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:14.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.837+0000 7f1009200640 1 -- 172.21.15.50:0/1525497721 shutdown_connections 2024-09-12T03:37:14.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.837+0000 7f1009200640 1 -- 172.21.15.50:0/1525497721 wait complete. 2024-09-12T03:37:14.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 Processor -- start 2024-09-12T03:37:14.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 -- start start 2024-09-12T03:37:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10041006a0 0x7f100419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1004101050 0x7f100419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f100419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1004113f50 con 0x7f10041006a0 2024-09-12T03:37:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1004113dd0 con 0x7f1004101050 2024-09-12T03:37:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.839+0000 7f1009200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10041140d0 con 0x7f100410e250 2024-09-12T03:37:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f100419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f100419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44046/0 (socket says 172.21.15.50:44046) 2024-09-12T03:37:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f0ffaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1004101050 0x7f100419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 -- 172.21.15.50:0/3162777973 learned_addr learned my addr 172.21.15.50:0/3162777973 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 -- 172.21.15.50:0/3162777973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1004101050 msgr2=0x7f100419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1004101050 0x7f100419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 -- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10041006a0 msgr2=0x7f100419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10041006a0 0x7f100419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f1003e00640 1 -- 172.21.15.50:0/3162777973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100419f3d0 con 0x7f100410e250 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.840+0000 7f0ffaa00640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1004101050 0x7f100419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.841+0000 7f1003e00640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f100419eca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009ee0 tx=0x7f0ffc00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.841+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00c880 con 0x7f100410e250 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.841+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ffc005020 con 0x7f100410e250 2024-09-12T03:37:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.841+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc004b80 con 0x7f100410e250 2024-09-12T03:37:14.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.842+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100419f6c0 con 0x7f100410e250 2024-09-12T03:37:14.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.842+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10041028b0 con 0x7f100410e250 2024-09-12T03:37:14.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.846+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1004100a80 con 0x7f100410e250 2024-09-12T03:37:14.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.847+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0ffc004d20 con 0x7f100410e250 2024-09-12T03:37:14.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.851+0000 7f1000c00640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0fe80a2cf0 0x7f0fe80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:14.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.851+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7f0ffc0d7e80 con 0x7f100410e250 2024-09-12T03:37:14.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.851+0000 7f1003400640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0fe80a2cf0 0x7f0fe80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:14.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.854+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ffc0a4770 con 0x7f100410e250 2024-09-12T03:37:14.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:14.856+0000 7f1003400640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0fe80a2cf0 0x7f0fe80a51b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009a60 tx=0x7f0ff4005f80 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:15.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.263+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 --> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f10040ffab0 con 0x7f0fe80a2cf0 2024-09-12T03:37:15.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.287+0000 7f1000c00640 1 -- 172.21.15.50:0/3162777973 <== mgr.25157 v2:172.21.15.50:6800/682249052 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f10040ffab0 con 0x7f0fe80a2cf0 2024-09-12T03:37:15.288 INFO:teuthology.orchestra.run.smithi050.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T03:27:12", "first": "2024-09-12T03:28:00", "last": "2024-09-12T03:37:00", "last_pruned": "2024-09-12T03:37:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-12T03:37:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.290+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0fe80a2cf0 msgr2=0x7f0fe80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.290+0000 7f1009200640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0fe80a2cf0 0x7f0fe80a51b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009a60 tx=0x7f0ff4005f80 comp rx=0 tx=0).stop 2024-09-12T03:37:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.290+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 msgr2=0x7f100419eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.290+0000 7f1009200640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f100419eca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009ee0 tx=0x7f0ffc00ed10 comp rx=0 tx=0).stop 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 shutdown_connections 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100410e250 0x7f100419eca0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1004101050 0x7f100419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f0fe80a2cf0 0x7f0fe80a51b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 --2- 172.21.15.50:0/3162777973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10041006a0 0x7f100419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 >> 172.21.15.50:0/3162777973 conn(0x7f10040777c0 msgr2=0x7f10040ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 shutdown_connections 2024-09-12T03:37:15.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.291+0000 7f1009200640 1 -- 172.21.15.50:0/3162777973 wait complete. 2024-09-12T03:37:15.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:14 smithi142 ceph-mon[35329]: pgmap v312: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:37:15.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:14 smithi050 ceph-mon[30378]: pgmap v312: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:37:15.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:37:15.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:14 smithi082 ceph-mon[35046]: pgmap v312: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:37:15.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 -- 172.21.15.50:0/3425937250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074590 msgr2=0x7f8988074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 --2- 172.21.15.50:0/3425937250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074590 0x7f8988074970 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8978009900 tx=0x7f897802a480 comp rx=0 tx=0).stop 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 -- 172.21.15.50:0/3425937250 shutdown_connections 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 --2- 172.21.15.50:0/3425937250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89881128c0 0x7f8988114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 --2- 172.21.15.50:0/3425937250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8988074eb0 0x7f8988112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 --2- 172.21.15.50:0/3425937250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074590 0x7f8988074970 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.518+0000 7f898d800640 1 -- 172.21.15.50:0/3425937250 >> 172.21.15.50:0/3425937250 conn(0x7f89880fd5d0 msgr2=0x7f89880ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.519+0000 7f898d800640 1 -- 172.21.15.50:0/3425937250 shutdown_connections 2024-09-12T03:37:15.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.519+0000 7f898d800640 1 -- 172.21.15.50:0/3425937250 wait complete. 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 Processor -- start 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 -- start start 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 0x7f89881a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 0x7f89881a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89881128c0 0x7f898819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89881182d0 con 0x7f89881128c0 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8988118150 con 0x7f8988074eb0 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.520+0000 7f898d800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8988118450 con 0x7f8988074590 2024-09-12T03:37:15.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 0x7f89881a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 0x7f89881a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 0x7f89881a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42712/0 (socket says 172.21.15.50:42712) 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 0x7f89881a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44074/0 (socket says 172.21.15.50:44074) 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987400640 1 -- 172.21.15.50:0/2303356957 learned_addr learned my addr 172.21.15.50:0/2303356957 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f898ce00640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89881128c0 0x7f898819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 -- 172.21.15.50:0/2303356957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 msgr2=0x7f89881a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 0x7f89881a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 -- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89881128c0 msgr2=0x7f898819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89881128c0 0x7f898819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 -- 172.21.15.50:0/2303356957 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898819ed70 con 0x7f8988074590 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987400640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 0x7f89881a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:15.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.521+0000 7f8987e00640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 0x7f89881a4440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f897802a9a0 tx=0x7f8978003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:15.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.522+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897803f070 con 0x7f8988074590 2024-09-12T03:37:15.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.522+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8978031c90 con 0x7f8988074590 2024-09-12T03:37:15.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.522+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978035030 con 0x7f8988074590 2024-09-12T03:37:15.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.522+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898819f000 con 0x7f8988074590 2024-09-12T03:37:15.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.522+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89881ab300 con 0x7f8988074590 2024-09-12T03:37:15.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.523+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8988074970 con 0x7f8988074590 2024-09-12T03:37:15.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.525+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8978042070 con 0x7f8988074590 2024-09-12T03:37:15.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.527+0000 7f8984c00640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f896c0a2d70 0x7f896c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.527+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7f89780f0150 con 0x7f8988074590 2024-09-12T03:37:15.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.527+0000 7f8987400640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f896c0a2d70 0x7f896c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.528+0000 7f8987400640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f896c0a2d70 0x7f896c0a5230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89880ffdf0 tx=0x7f8974005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:15.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.531+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89780bc280 con 0x7f8988074590 2024-09-12T03:37:15.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.771+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 --> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8988062630 con 0x7f896c0a2d70 2024-09-12T03:37:15.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.778+0000 7f8984c00640 1 -- 172.21.15.50:0/2303356957 <== mgr.25157 v2:172.21.15.50:6800/682249052 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8988062630 con 0x7f896c0a2d70 2024-09-12T03:37:15.779 INFO:teuthology.orchestra.run.smithi050.stdout:/ 1m 6m3h 2024-09-12T03:37:15.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f896c0a2d70 msgr2=0x7f896c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f896c0a2d70 0x7f896c0a5230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89880ffdf0 tx=0x7f8974005f90 comp rx=0 tx=0).stop 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 msgr2=0x7f89881a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 0x7f89881a4440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f897802a9a0 tx=0x7f8978003b20 comp rx=0 tx=0).stop 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 shutdown_connections 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f896c0a2d70 0x7f896c0a5230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89881128c0 0x7f898819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8988074eb0 0x7f89881a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 --2- 172.21.15.50:0/2303356957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8988074590 0x7f89881a4440 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 >> 172.21.15.50:0/2303356957 conn(0x7f89880fd5d0 msgr2=0x7f89880fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 shutdown_connections 2024-09-12T03:37:15.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.781+0000 7f898d800640 1 -- 172.21.15.50:0/2303356957 wait complete. 2024-09-12T03:37:15.799 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:37:15.810 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 -- 172.21.15.50:0/1971160243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 msgr2=0x7f4664101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 --2- 172.21.15.50:0/1971160243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664101d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4650009890 tx=0x7f465002a360 comp rx=0 tx=0).stop 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 -- 172.21.15.50:0/1971160243 shutdown_connections 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 --2- 172.21.15.50:0/1971160243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 0x7f466410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 --2- 172.21.15.50:0/1971160243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664101d40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 --2- 172.21.15.50:0/1971160243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46641078f0 0x7f4664107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 -- 172.21.15.50:0/1971160243 >> 172.21.15.50:0/1971160243 conn(0x7f46640fd5e0 msgr2=0x7f46640ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:15.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 -- 172.21.15.50:0/1971160243 shutdown_connections 2024-09-12T03:37:15.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.972+0000 7f466b000640 1 -- 172.21.15.50:0/1971160243 wait complete. 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 Processor -- start 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 -- start start 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 0x7f4664198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46641078f0 0x7f466419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4664111d30 con 0x7f46641018e0 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4664111bb0 con 0x7f4664102280 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.973+0000 7f466b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4664111eb0 con 0x7f46641078f0 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47372/0 (socket says 172.21.15.50:47372) 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 -- 172.21.15.50:0/2000553850 learned_addr learned my addr 172.21.15.50:0/2000553850 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f466a600640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46641078f0 0x7f466419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669200640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 0x7f4664198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 -- 172.21.15.50:0/2000553850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46641078f0 msgr2=0x7f466419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46641078f0 0x7f466419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 -- 172.21.15.50:0/2000553850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 msgr2=0x7f4664198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 0x7f4664198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669c00640 1 -- 172.21.15.50:0/2000553850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466419d230 con 0x7f46641018e0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.974+0000 7f4669200640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 0x7f4664198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.975+0000 7f4669c00640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664198240 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f465800e9f0 tx=0x7f465800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.975+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800cd80 con 0x7f46641018e0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.975+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4658005500 con 0x7f46641018e0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.975+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466419d520 con 0x7f46641018e0 2024-09-12T03:37:15.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.975+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658010690 con 0x7f46641018e0 2024-09-12T03:37:15.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.975+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a4db0 con 0x7f46641018e0 2024-09-12T03:37:15.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.976+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4664101d40 con 0x7f46641018e0 2024-09-12T03:37:15.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.978+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f46580046c0 con 0x7f46641018e0 2024-09-12T03:37:15.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.979+0000 7f4662000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f464c0a2d00 0x7f464c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:15.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.979+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7f46580cfc80 con 0x7f46641018e0 2024-09-12T03:37:15.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.979+0000 7f4669200640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f464c0a2d00 0x7f464c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:15.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.980+0000 7f4669200640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f464c0a2d00 0x7f464c0a51c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4650000c00 tx=0x7f4650005890 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:15.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:15.984+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f465809c400 con 0x7f46641018e0 2024-09-12T03:37:16.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.172+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46641098c0 con 0x7f46641018e0 2024-09-12T03:37:16.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.174+0000 7f4662000640 1 -- 172.21.15.50:0/2000553850 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25514 (secure 0 0 0) 0x7f465809bd00 con 0x7f46641018e0 2024-09-12T03:37:16.175 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:37:16.176 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":103,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:37:10.891843+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3692206583":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/308265429":"2024-09-13T03:27:11.145278+0000","172.21.15.50:6801/63385930":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1993742081":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/3889049657":"2024-09-13T03:27:11.145278+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/63385930":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f464c0a2d00 msgr2=0x7f464c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f464c0a2d00 0x7f464c0a51c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4650000c00 tx=0x7f4650005890 comp rx=0 tx=0).stop 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 msgr2=0x7f4664198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664198240 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f465800e9f0 tx=0x7f465800edf0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 shutdown_connections 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46641078f0 0x7f466419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4664102280 0x7f4664198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f464c0a2d00 0x7f464c0a51c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 --2- 172.21.15.50:0/2000553850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46641018e0 0x7f4664198240 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 >> 172.21.15.50:0/2000553850 conn(0x7f46640fd5e0 msgr2=0x7f46640fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 shutdown_connections 2024-09-12T03:37:16.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.177+0000 7f466b000640 1 -- 172.21.15.50:0/2000553850 wait complete. 2024-09-12T03:37:16.195 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:19:33.283337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:24:28.237350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:24:30.608939+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-12T03:24:39.200529+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-12T03:37:16.195 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 -- 172.21.15.50:0/1433354800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4103f10 msgr2=0x7efea41042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 --2- 172.21.15.50:0/1433354800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4103f10 0x7efea41042f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe94009900 tx=0x7efe9402a400 comp rx=0 tx=0).stop 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 -- 172.21.15.50:0/1433354800 shutdown_connections 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 --2- 172.21.15.50:0/1433354800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efea4109060 0x7efea4110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 --2- 172.21.15.50:0/1433354800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea41048c0 0x7efea4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 --2- 172.21.15.50:0/1433354800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4103f10 0x7efea41042f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 -- 172.21.15.50:0/1433354800 >> 172.21.15.50:0/1433354800 conn(0x7efea40ff940 msgr2=0x7efea4101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:16.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 -- 172.21.15.50:0/1433354800 shutdown_connections 2024-09-12T03:37:16.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.356+0000 7efeaac00640 1 -- 172.21.15.50:0/1433354800 wait complete. 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 Processor -- start 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 -- start start 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 0x7efea419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4109060 0x7efea419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efea419eef0 0x7efea419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efea4114100 con 0x7efea4103f10 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efea4113f80 con 0x7efea419eef0 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.357+0000 7efeaac00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efea4114280 con 0x7efea4109060 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efea3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 0x7efea419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efea3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 0x7efea419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47390/0 (socket says 172.21.15.50:47390) 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4109060 0x7efea419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efea3e00640 1 -- 172.21.15.50:0/1631061672 learned_addr learned my addr 172.21.15.50:0/1631061672 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efea8c00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efea419eef0 0x7efea419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 -- 172.21.15.50:0/1631061672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efea419eef0 msgr2=0x7efea419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efea419eef0 0x7efea419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 -- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 msgr2=0x7efea419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 0x7efea419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 -- 172.21.15.50:0/1631061672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea419f9e0 con 0x7efea4109060 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efea3e00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 0x7efea419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.358+0000 7efe9be00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4109060 0x7efea419ab70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00d6a0 tx=0x7efe8c00db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.359+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c017070 con 0x7efea4109060 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.359+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8c009820 con 0x7efea4109060 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.359+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c004d00 con 0x7efea4109060 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.359+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea419fcd0 con 0x7efea4109060 2024-09-12T03:37:16.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.359+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea41a7470 con 0x7efea4109060 2024-09-12T03:37:16.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.360+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea41042f0 con 0x7efea4109060 2024-09-12T03:37:16.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.362+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efe8c0099d0 con 0x7efea4109060 2024-09-12T03:37:16.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.363+0000 7efea1600640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efe880a2d80 0x7efe880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.363+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7efe8c0cf140 con 0x7efea4109060 2024-09-12T03:37:16.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.363+0000 7efea3e00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efe880a2d80 0x7efe880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.364+0000 7efea3e00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efe880a2d80 0x7efe880a5240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe940096c0 tx=0x7efe940046c0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:16.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.367+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe8c09b270 con 0x7efea4109060 2024-09-12T03:37:16.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.553+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efea41048c0 con 0x7efea4109060 2024-09-12T03:37:16.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.554+0000 7efea1600640 1 -- 172.21.15.50:0/1631061672 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7efe8c09f570 con 0x7efea4109060 2024-09-12T03:37:16.555 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-09-12T03:37:16.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.556+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efe880a2d80 msgr2=0x7efe880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.556+0000 7efeaac00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efe880a2d80 0x7efe880a5240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe940096c0 tx=0x7efe940046c0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.556+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4109060 msgr2=0x7efea419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.556+0000 7efeaac00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4109060 0x7efea419ab70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00d6a0 tx=0x7efe8c00db60 comp rx=0 tx=0).stop 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 shutdown_connections 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efea419eef0 0x7efea419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea4109060 0x7efea419ab70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7efe880a2d80 0x7efe880a5240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 --2- 172.21.15.50:0/1631061672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efea4103f10 0x7efea419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 >> 172.21.15.50:0/1631061672 conn(0x7efea40ff940 msgr2=0x7efea4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 shutdown_connections 2024-09-12T03:37:16.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.557+0000 7efeaac00640 1 -- 172.21.15.50:0/1631061672 wait complete. 2024-09-12T03:37:16.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:37:16.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 -- 172.21.15.50:0/4058091039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc103df0 msgr2=0x7f65cc1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 --2- 172.21.15.50:0/4058091039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc103df0 0x7f65cc1041d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009860 tx=0x7f65c002a340 comp rx=0 tx=0).stop 2024-09-12T03:37:16.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 -- 172.21.15.50:0/4058091039 shutdown_connections 2024-09-12T03:37:16.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 --2- 172.21.15.50:0/4058091039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc108f40 0x7f65cc110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 --2- 172.21.15.50:0/4058091039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc1047a0 0x7f65cc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 --2- 172.21.15.50:0/4058091039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc103df0 0x7f65cc1041d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 -- 172.21.15.50:0/4058091039 >> 172.21.15.50:0/4058091039 conn(0x7f65cc0ff840 msgr2=0x7f65cc101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:16.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 -- 172.21.15.50:0/4058091039 shutdown_connections 2024-09-12T03:37:16.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.735+0000 7f65d2e00640 1 -- 172.21.15.50:0/4058091039 wait complete. 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.736+0000 7f65d2e00640 1 Processor -- start 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.736+0000 7f65d2e00640 1 -- start start 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc103df0 0x7f65cc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 0x7f65cc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 0x7f65cc19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc113fa0 con 0x7f65cc108f40 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc113e20 con 0x7f65cc19ed80 2024-09-12T03:37:16.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc114120 con 0x7f65cc103df0 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 0x7f65cc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc103df0 0x7f65cc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 0x7f65cc19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc103df0 0x7f65cc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44134/0 (socket says 172.21.15.50:44134) 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 0x7f65cc19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47398/0 (socket says 172.21.15.50:47398) 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d1a00640 1 -- 172.21.15.50:0/3765733234 learned_addr learned my addr 172.21.15.50:0/3765733234 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 0x7f65cc19f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.50:42770/0 (socket says 172.21.15.50:42770) 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 -- 172.21.15.50:0/3765733234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc103df0 msgr2=0x7f65cc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc103df0 0x7f65cc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 -- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 msgr2=0x7f65cc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 0x7f65cc19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d2400640 1 -- 172.21.15.50:0/3765733234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc19f870 con 0x7f65cc19ed80 2024-09-12T03:37:16.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.737+0000 7f65d1000640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 0x7f65cc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.738+0000 7f65d2400640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 0x7f65cc19f230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009fd0 tx=0x7f65c400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.738+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c400cd60 con 0x7f65cc19ed80 2024-09-12T03:37:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.738+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c4005020 con 0x7f65cc19ed80 2024-09-12T03:37:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.738+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc19fb60 con 0x7f65cc19ed80 2024-09-12T03:37:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.738+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4010690 con 0x7f65cc19ed80 2024-09-12T03:37:16.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.738+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1a7350 con 0x7f65cc19ed80 2024-09-12T03:37:16.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.739+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc1041d0 con 0x7f65cc19ed80 2024-09-12T03:37:16.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.741+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f65c4005540 con 0x7f65cc19ed80 2024-09-12T03:37:16.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.742+0000 7f65ca000640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f65b40a2d00 0x7f65b40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:16.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.743+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7f65c40cfe40 con 0x7f65cc19ed80 2024-09-12T03:37:16.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.743+0000 7f65d1a00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f65b40a2d00 0x7f65b40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:16.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.743+0000 7f65d1a00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f65b40a2d00 0x7f65b40a51c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65c00095f0 tx=0x7f65c00056d0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:16.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.745+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65c409c5c0 con 0x7f65cc19ed80 2024-09-12T03:37:16.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.929+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f65cc1047a0 con 0x7f65cc19ed80 2024-09-12T03:37:16.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.930+0000 7f65ca000640 1 -- 172.21.15.50:0/3765733234 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f65c409bec0 con 0x7f65cc19ed80 2024-09-12T03:37:16.930 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f65b40a2d00 msgr2=0x7f65b40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f65b40a2d00 0x7f65b40a51c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65c00095f0 tx=0x7f65c00056d0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 msgr2=0x7f65cc19f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 0x7f65cc19f230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009fd0 tx=0x7f65c400ee50 comp rx=0 tx=0).stop 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 shutdown_connections 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65cc19ed80 0x7f65cc19f230 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f65b40a2d00 0x7f65b40a51c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65cc108f40 0x7f65cc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 --2- 172.21.15.50:0/3765733234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc103df0 0x7f65cc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:16.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 >> 172.21.15.50:0/3765733234 conn(0x7f65cc0ff840 msgr2=0x7f65cc100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:16.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 shutdown_connections 2024-09-12T03:37:16.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:16.932+0000 7f65d2e00640 1 -- 172.21.15.50:0/3765733234 wait complete. 2024-09-12T03:37:16.949 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:37:17.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 -- 172.21.15.50:0/2147867137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 msgr2=0x7f76e8104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 --2- 172.21.15.50:0/2147867137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e8104250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009900 tx=0x7f76d802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 -- 172.21.15.50:0/2147867137 shutdown_connections 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 --2- 172.21.15.50:0/2147867137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 0x7f76e8110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 --2- 172.21.15.50:0/2147867137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e8104820 0x7f76e8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 --2- 172.21.15.50:0/2147867137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e8104250 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 -- 172.21.15.50:0/2147867137 >> 172.21.15.50:0/2147867137 conn(0x7f76e80ff8a0 msgr2=0x7f76e8101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 -- 172.21.15.50:0/2147867137 shutdown_connections 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.110+0000 7f76eca00640 1 -- 172.21.15.50:0/2147867137 wait complete. 2024-09-12T03:37:17.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 Processor -- start 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 -- start start 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 0x7f76e819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e819ee00 0x7f76e819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8114070 con 0x7f76e819ee00 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8113ef0 con 0x7f76e8108fc0 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.111+0000 7f76eca00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81141f0 con 0x7f76e8103e70 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e819ee00 0x7f76e819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:44152/0 (socket says 172.21.15.50:44152) 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 -- 172.21.15.50:0/1612607205 learned_addr learned my addr 172.21.15.50:0/1612607205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76dde00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 0x7f76e819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 -- 172.21.15.50:0/1612607205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 msgr2=0x7f76e819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 0x7f76e819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 -- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e819ee00 msgr2=0x7f76e819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e819ee00 0x7f76e819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 -- 172.21.15.50:0/1612607205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e819f8f0 con 0x7f76e8103e70 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76dde00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 0x7f76e819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e7200640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e819ee00 0x7f76e819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.112+0000 7f76e6800640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e819a540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76d8004770 tx=0x7f76d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.113+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d803f070 con 0x7f76e8103e70 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.113+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e819fb80 con 0x7f76e8103e70 2024-09-12T03:37:17.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.113+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d8031ca0 con 0x7f76e8103e70 2024-09-12T03:37:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.113+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8035030 con 0x7f76e8103e70 2024-09-12T03:37:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.113+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a7340 con 0x7f76e8103e70 2024-09-12T03:37:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.114+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8104250 con 0x7f76e8103e70 2024-09-12T03:37:17.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.116+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f76d8042070 con 0x7f76e8103e70 2024-09-12T03:37:17.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.117+0000 7f76dfe00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f76cc0a2d00 0x7f76cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.117+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7f76d80efc10 con 0x7f76e8103e70 2024-09-12T03:37:17.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.117+0000 7f76dde00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f76cc0a2d00 0x7f76cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.118+0000 7f76dde00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f76cc0a2d00 0x7f76cc0a51c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76e80ffcc0 tx=0x7f76d0005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:17.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.121+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76d80bc450 con 0x7f76e8103e70 2024-09-12T03:37:17.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:16 smithi142 ceph-mon[35329]: from='client.25774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:37:17.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:16 smithi142 ceph-mon[35329]: pgmap v313: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:37:17.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:16 smithi142 ceph-mon[35329]: from='client.25780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:37:17.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:16 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2000553850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:37:17.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:16 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1631061672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:37:17.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.308+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f76e810c170 con 0x7f76e8103e70 2024-09-12T03:37:17.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.309+0000 7f76dfe00640 1 -- 172.21.15.50:0/1612607205 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f76d80bbd50 con 0x7f76e8103e70 2024-09-12T03:37:17.310 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-09-12T03:37:17.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.311+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f76cc0a2d00 msgr2=0x7f76cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.311+0000 7f76eca00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f76cc0a2d00 0x7f76cc0a51c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76e80ffcc0 tx=0x7f76d0005f90 comp rx=0 tx=0).stop 2024-09-12T03:37:17.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.311+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 msgr2=0x7f76e819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.311+0000 7f76eca00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e819a540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76d8004770 tx=0x7f76d8003c10 comp rx=0 tx=0).stop 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 shutdown_connections 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f76cc0a2d00 0x7f76cc0a51c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e819ee00 0x7f76e819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8108fc0 0x7f76e819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 --2- 172.21.15.50:0/1612607205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e8103e70 0x7f76e819a540 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 >> 172.21.15.50:0/1612607205 conn(0x7f76e80ff8a0 msgr2=0x7f76e8100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 shutdown_connections 2024-09-12T03:37:17.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.312+0000 7f76eca00640 1 -- 172.21.15.50:0/1612607205 wait complete. 2024-09-12T03:37:17.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:16 smithi050 ceph-mon[30378]: from='client.25774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:37:17.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:16 smithi050 ceph-mon[30378]: pgmap v313: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:37:17.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:16 smithi050 ceph-mon[30378]: from='client.25780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:37:17.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:16 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2000553850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:37:17.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:16 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1631061672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:37:17.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-12T03:37:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:16 smithi082 ceph-mon[35046]: from='client.25774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:37:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:16 smithi082 ceph-mon[35046]: pgmap v313: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 139 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-12T03:37:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:16 smithi082 ceph-mon[35046]: from='client.25780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:37:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:16 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2000553850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:37:17.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:16 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1631061672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:37:17.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 -- 172.21.15.50:0/1524010992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 msgr2=0x7f8544110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 --2- 172.21.15.50:0/1524010992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f8544110a90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f853c009be0 tx=0x7f853c02a6c0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 -- 172.21.15.50:0/1524010992 shutdown_connections 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 --2- 172.21.15.50:0/1524010992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f8544110a90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 --2- 172.21.15.50:0/1524010992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85441047a0 0x7f85441087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 --2- 172.21.15.50:0/1524010992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f85441041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 -- 172.21.15.50:0/1524010992 >> 172.21.15.50:0/1524010992 conn(0x7f85440ff840 msgr2=0x7f8544101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 -- 172.21.15.50:0/1524010992 shutdown_connections 2024-09-12T03:37:17.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.491+0000 7f8543200640 1 -- 172.21.15.50:0/1524010992 wait complete. 2024-09-12T03:37:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.492+0000 7f8543200640 1 Processor -- start 2024-09-12T03:37:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.492+0000 7f8543200640 1 -- start start 2024-09-12T03:37:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8543200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f854419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8543200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f854419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8543200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f854419ed10 0x7f854419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8543200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8544113fa0 con 0x7f8544103df0 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8543200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8544113e20 con 0x7f8544108f40 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8543200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8544114120 con 0x7f854419ed10 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f854419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f854419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47426/0 (socket says 172.21.15.50:47426) 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541e00640 1 -- 172.21.15.50:0/145267479 learned_addr learned my addr 172.21.15.50:0/145267479 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f854419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8542800640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f854419ed10 0x7f854419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541400640 1 -- 172.21.15.50:0/145267479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f854419ed10 msgr2=0x7f854419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541400640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f854419ed10 0x7f854419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541400640 1 -- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 msgr2=0x7f854419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541400640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f854419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541400640 1 -- 172.21.15.50:0/145267479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854419f800 con 0x7f8544108f40 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.493+0000 7f8541e00640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f854419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.494+0000 7f8541400640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f854419a990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f853000e9c0 tx=0x7f853000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.494+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853000ccc0 con 0x7f8544108f40 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.494+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8530010910 con 0x7f8544108f40 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.494+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854419faf0 con 0x7f8544108f40 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.494+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300188c0 con 0x7f8544108f40 2024-09-12T03:37:17.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.494+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441a7280 con 0x7f8544108f40 2024-09-12T03:37:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.495+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85441041d0 con 0x7f8544108f40 2024-09-12T03:37:17.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.497+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8530010ac0 con 0x7f8544108f40 2024-09-12T03:37:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.498+0000 7f853aa00640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f85280a2d00 0x7f85280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.499+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7f85300d00a0 con 0x7f8544108f40 2024-09-12T03:37:17.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.499+0000 7f8541e00640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f85280a2d00 0x7f85280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.499+0000 7f8541e00640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f85280a2d00 0x7f85280a51c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85340096c0 tx=0x7f8534009260 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:17.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.501+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f853009c1d0 con 0x7f8544108f40 2024-09-12T03:37:17.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.686+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f85441047a0 con 0x7f8544108f40 2024-09-12T03:37:17.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.687+0000 7f853aa00640 1 -- 172.21.15.50:0/145267479 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7f85300a04d0 con 0x7f8544108f40 2024-09-12T03:37:17.689 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f85280a2d00 msgr2=0x7f85280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f85280a2d00 0x7f85280a51c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85340096c0 tx=0x7f8534009260 comp rx=0 tx=0).stop 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 msgr2=0x7f854419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f854419a990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f853000e9c0 tx=0x7f853000ee80 comp rx=0 tx=0).stop 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 shutdown_connections 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f854419ed10 0x7f854419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8544108f40 0x7f854419a990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f85280a2d00 0x7f85280a51c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 --2- 172.21.15.50:0/145267479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8544103df0 0x7f854419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 >> 172.21.15.50:0/145267479 conn(0x7f85440ff840 msgr2=0x7f8544100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 shutdown_connections 2024-09-12T03:37:17.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.689+0000 7f8543200640 1 -- 172.21.15.50:0/145267479 wait complete. 2024-09-12T03:37:17.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:37:17.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 -- 172.21.15.50:0/422958301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 msgr2=0x7fd0dc0697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/422958301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc0697c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009900 tx=0x7fd0c802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 -- 172.21.15.50:0/422958301 shutdown_connections 2024-09-12T03:37:17.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/422958301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0dc069d00 0x7fd0dc10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/422958301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc0697c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/422958301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc1078f0 0x7fd0dc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 -- 172.21.15.50:0/422958301 >> 172.21.15.50:0/422958301 conn(0x7fd0dc076550 msgr2=0x7fd0dc076960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:17.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 -- 172.21.15.50:0/422958301 shutdown_connections 2024-09-12T03:37:17.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.863+0000 7fd0dbe00640 1 -- 172.21.15.50:0/422958301 wait complete. 2024-09-12T03:37:17.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.864+0000 7fd0dbe00640 1 Processor -- start 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.864+0000 7fd0dbe00640 1 -- start start 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0dbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0dc069d00 0x7fd0dc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0dbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc1078f0 0x7fd0dc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0dbe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111d20 con 0x7fd0dc069380 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0dbe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111ba0 con 0x7fd0dc1078f0 2024-09-12T03:37:17.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0dbe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111ea0 con 0x7fd0dc069d00 2024-09-12T03:37:17.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47436/0 (socket says 172.21.15.50:47436) 2024-09-12T03:37:17.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 -- 172.21.15.50:0/4081246601 learned_addr learned my addr 172.21.15.50:0/4081246601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:17.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0da000640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0dc069d00 0x7fd0dc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0db400640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc1078f0 0x7fd0dc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 -- 172.21.15.50:0/4081246601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0dc069d00 msgr2=0x7fd0dc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0dc069d00 0x7fd0dc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 -- 172.21.15.50:0/4081246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc1078f0 msgr2=0x7fd0dc19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc1078f0 0x7fd0dc19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.865+0000 7fd0daa00640 1 -- 172.21.15.50:0/4081246601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc19f390 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.866+0000 7fd0daa00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc19a3a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00ea60 tx=0x7fd0cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.866+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00cde0 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.866+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0cc0035f0 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.866+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc19f680 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.866+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc010670 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.866+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc1a6f10 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.867+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc0697c0 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.869+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd0cc0037a0 con 0x7fd0dc069380 2024-09-12T03:37:17.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.870+0000 7fd0d3400640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd0c00a2d80 0x7fd0c00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:17.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.871+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9613+0+0 (secure 0 0 0) 0x7fd0cc0cfa60 con 0x7fd0dc069380 2024-09-12T03:37:17.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.871+0000 7fd0da000640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd0c00a2d80 0x7fd0c00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:17.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.871+0000 7fd0da000640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd0c00a2d80 0x7fd0c00a5240 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000c00 tx=0x7fd0c8005b50 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:17.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:17.874+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0cc09c1e0 con 0x7fd0dc069380 2024-09-12T03:37:18.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.112+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd0dc1a71b0 con 0x7fd0dc069380 2024-09-12T03:37:18.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.122+0000 7fd0d3400640 1 -- 172.21.15.50:0/4081246601 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740896 (secure 0 0 0) 0x7fd0cc09bae0 con 0x7fd0dc069380 2024-09-12T03:37:18.134 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:37:18.134 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:37:18.134 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 26, 2024-09-12T03:37:18.134 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.134 INFO:teuthology.orchestra.run.smithi050.stdout: "active_gid": 25157, 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "active_name": "x", 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addrs": { 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6800", 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 682249052 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6801", 2024-09-12T03:37:18.135 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 682249052 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addr": "172.21.15.50:6801/682249052", 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "active_change": "2024-09-12T03:27:11.145532+0000", 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "standbys": [ 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 25147, 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "z", 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:37:18.136 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.137 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.138 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.139 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.140 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.141 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.141 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.141 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.141 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.141 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.143 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.143 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.143 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.143 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.143 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.143 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.144 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.144 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.144 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.144 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.144 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.144 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.145 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.145 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.145 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.145 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.145 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.145 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.146 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.146 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.146 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.146 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.146 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.146 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.147 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.147 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-09-12T03:37:18.147 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-09-12T03:37:18.147 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.147 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.147 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.148 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.148 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.148 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.148 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.148 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:37:18.149 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.149 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.149 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.149 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-09-12T03:37:18.149 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-09-12T03:37:18.150 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.150 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.150 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.150 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-09-12T03:37:18.150 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.150 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.151 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.151 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-09-12T03:37:18.151 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.151 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.151 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.152 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-09-12T03:37:18.152 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-09-12T03:37:18.152 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.152 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.152 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.152 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.153 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.153 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.153 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.153 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-09-12T03:37:18.153 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.153 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.154 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.155 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.155 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.155 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.155 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.155 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-09-12T03:37:18.156 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.156 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.156 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.156 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.156 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-09-12T03:37:18.156 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-09-12T03:37:18.158 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-09-12T03:37:18.159 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.159 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.159 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.159 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.159 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.159 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.160 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.160 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:37:18.160 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.160 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.160 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.161 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.161 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-09-12T03:37:18.161 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-09-12T03:37:18.161 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.161 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:37:18.163 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.163 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.163 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.163 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.163 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-09-12T03:37:18.163 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-09-12T03:37:18.164 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.164 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.164 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.164 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.164 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.164 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.165 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.165 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-09-12T03:37:18.165 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.165 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.165 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.166 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.166 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.166 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.166 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.167 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-09-12T03:37:18.167 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.167 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.167 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.167 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-09-12T03:37:18.167 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-09-12T03:37:18.168 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.168 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.168 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.168 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.168 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.169 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.169 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.169 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:37:18.169 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.169 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.169 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.170 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-09-12T03:37:18.170 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-09-12T03:37:18.170 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.170 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.171 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.171 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-09-12T03:37:18.171 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.171 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.171 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.171 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:37:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:37:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.172 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-09-12T03:37:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-09-12T03:37:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-09-12T03:37:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-09-12T03:37:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.174 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:37:18.175 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:37:18.175 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.175 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.175 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-09-12T03:37:18.175 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:37:18.176 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.176 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.176 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.176 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-09-12T03:37:18.176 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.177 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-09-12T03:37:18.177 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.177 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:37:18.177 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.177 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-09-12T03:37:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-09-12T03:37:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.178 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.179 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.179 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:37:18.179 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.179 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.179 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.179 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:37:18.180 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:37:18.180 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.180 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.180 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.180 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-09-12T03:37:18.181 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-09-12T03:37:18.181 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.181 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.181 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.181 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.182 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.183 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.183 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-09-12T03:37:18.183 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-09-12T03:37:18.183 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.183 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:37:18.184 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.185 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.185 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.185 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-09-12T03:37:18.185 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-09-12T03:37:18.185 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.186 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.186 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.186 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.186 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-09-12T03:37:18.186 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-09-12T03:37:18.186 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:37:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:37:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.188 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.188 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.188 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.188 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.188 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.189 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.190 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.191 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.192 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.193 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.194 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.195 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-09-12T03:37:18.196 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:37:18.197 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:37:18.198 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.199 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.200 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.201 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.202 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:37:18.203 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.204 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.205 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.206 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-09-12T03:37:18.207 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.208 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.209 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.210 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.211 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.212 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.213 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:37:18.214 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.215 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.216 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.217 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.218 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:37:18.219 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.220 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:37:18.221 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.222 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.223 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.225 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.226 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.227 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.228 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.229 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.230 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.231 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.232 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.233 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-09-12T03:37:18.234 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-09-12T03:37:18.235 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.238 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.239 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-09-12T03:37:18.240 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-09-12T03:37:18.241 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.242 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.243 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.244 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.245 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.246 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.247 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.252 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.253 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:37:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.256 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.257 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.258 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.259 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.260 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-09-12T03:37:18.261 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-09-12T03:37:18.262 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-09-12T03:37:18.263 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.264 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.265 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.266 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.267 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.268 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.269 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.270 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.271 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.272 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.274 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.275 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.276 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.277 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.278 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.280 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.280 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.280 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.280 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.280 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-09-12T03:37:18.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.281 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.282 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.283 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.284 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.285 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.286 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.287 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.287 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.287 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.287 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.287 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.287 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.288 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:37:18.288 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:37:18.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.288 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.288 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.289 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.290 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.290 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.290 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.290 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:37:18.290 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:37:18.290 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.291 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.291 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.291 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.291 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.291 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.291 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.292 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.292 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.292 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.292 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.292 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.292 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:37:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:37:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.294 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.294 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.294 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.294 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.294 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.295 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.296 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.296 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.296 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.296 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.296 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.296 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.297 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.298 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.299 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:37:18.300 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.301 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.302 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.303 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.304 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.305 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.306 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.307 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.307 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.307 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.307 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.307 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.307 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.308 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.308 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.308 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.308 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.308 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-09-12T03:37:18.308 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.309 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:37:18.310 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.311 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.312 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.312 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.312 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.312 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:37:18.312 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:37:18.312 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.313 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.313 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.313 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.313 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.313 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.313 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.314 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.314 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.314 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.314 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.314 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.314 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.315 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.317 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.318 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.319 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.320 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.321 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.322 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.323 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.324 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.325 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.326 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.327 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:37:18.328 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.329 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.330 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.331 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:37:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:37:18.333 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.334 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.335 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-09-12T03:37:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.338 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.339 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.340 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.341 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.342 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:37:18.343 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.344 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:37:18.345 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.346 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.347 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.347 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.347 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.347 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:37:18.347 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.349 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.350 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.351 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-09-12T03:37:18.352 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.353 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.354 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.355 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.356 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.357 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.358 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.359 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.360 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.361 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-09-12T03:37:18.362 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.363 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.364 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.365 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.366 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.368 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-09-12T03:37:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-09-12T03:37:18.370 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.371 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.372 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.373 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.374 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.375 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.376 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.377 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-09-12T03:37:18.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.380 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.381 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.382 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.383 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-09-12T03:37:18.384 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.385 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.386 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.388 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-09-12T03:37:18.391 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-09-12T03:37:18.392 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.393 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-09-12T03:37:18.394 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.395 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:37:18.396 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.397 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-09-12T03:37:18.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.399 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.400 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.401 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.402 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.404 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.405 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.407 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.408 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.409 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.410 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.411 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.412 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.413 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.414 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.415 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.416 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.417 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.418 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.419 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.420 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.421 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-09-12T03:37:18.422 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.423 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.424 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.425 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.426 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.427 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.428 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.429 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.430 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.431 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.432 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.433 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.434 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.435 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.436 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.437 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-09-12T03:37:18.438 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.439 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.441 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.442 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.443 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.444 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.445 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.447 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.448 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.449 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.450 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.451 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.452 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.453 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.455 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.456 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.457 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.458 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.459 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.460 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.461 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.462 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.463 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.464 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.465 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-09-12T03:37:18.466 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.467 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.468 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.469 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.470 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.471 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.473 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.474 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.475 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.476 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.478 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.479 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.480 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.481 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.482 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.483 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-09-12T03:37:18.484 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.485 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.488 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.489 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.490 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.491 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.492 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.493 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:37:18.494 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-09-12T03:37:18.495 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-09-12T03:37:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.497 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.498 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.499 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.500 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.501 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.503 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.504 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.505 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.506 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.507 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.508 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-09-12T03:37:18.509 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-09-12T03:37:18.510 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.511 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.512 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.513 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.514 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.515 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.516 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.517 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.518 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.519 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.520 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.521 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.522 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.523 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.524 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.525 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.527 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.528 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.529 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.530 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.531 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.532 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.533 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.534 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.535 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.536 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.537 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.538 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.539 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.541 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.542 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.543 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.544 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.545 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.546 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.547 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.548 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-09-12T03:37:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-09-12T03:37:18.550 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.551 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-09-12T03:37:18.552 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-09-12T03:37:18.553 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-09-12T03:37:18.554 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-09-12T03:37:18.555 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.556 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.558 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.559 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.560 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-09-12T03:37:18.561 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-09-12T03:37:18.562 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.563 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.564 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.565 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.566 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.567 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.568 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.569 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.570 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.571 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.572 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.573 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.574 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.575 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.577 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.578 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.579 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-09-12T03:37:18.580 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.582 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.583 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.584 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.585 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.586 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.587 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.589 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.590 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.591 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.592 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.593 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.594 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.595 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.596 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.597 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.598 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.599 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.600 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.601 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.602 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.604 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.605 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.606 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.608 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.610 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.611 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.612 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.613 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.614 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.615 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.616 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.617 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.618 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.619 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.620 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.621 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.622 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.623 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.624 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.625 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-09-12T03:37:18.626 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.627 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.628 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.629 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.630 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.631 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.633 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.634 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.635 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-09-12T03:37:18.636 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:37:18.637 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.638 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 25151, 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "y", 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.639 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.640 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.641 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.642 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.644 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-09-12T03:37:18.645 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-09-12T03:37:18.646 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-09-12T03:37:18.647 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.648 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.652 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.653 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.654 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.655 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.656 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.657 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.658 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-09-12T03:37:18.659 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.660 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.661 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.662 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.663 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.664 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.665 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.666 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.667 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:37:18.668 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-09-12T03:37:18.669 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.670 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.673 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.674 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:37:18.675 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.676 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.677 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.678 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.679 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-09-12T03:37:18.680 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.681 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.684 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.685 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.686 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.687 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.688 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.690 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.691 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:37:18.692 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.693 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.694 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.695 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.696 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.697 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-09-12T03:37:18.698 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-09-12T03:37:18.699 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-09-12T03:37:18.700 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.701 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.702 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.703 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.704 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.705 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.707 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.708 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.709 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.710 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.711 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.712 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.713 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.714 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.715 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.716 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.717 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.718 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.719 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.720 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-09-12T03:37:18.721 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.722 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.723 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.724 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-09-12T03:37:18.725 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.726 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.728 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.729 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.730 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:37:18.731 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.732 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.733 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.734 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.735 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.736 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.737 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.738 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.739 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.740 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.741 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.742 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.743 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.744 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.745 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.746 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.748 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.749 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.750 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.751 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.752 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:37:18.753 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:37:18.754 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:37:18.755 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:37:18.756 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:37:18.757 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:37:18.758 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.759 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.760 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.761 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.762 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.763 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.764 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.765 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.766 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.767 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.768 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.770 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.771 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.772 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.773 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.774 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.775 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.776 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.777 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.778 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:37:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:37:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.783 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.784 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.785 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.786 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.787 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.788 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.789 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.790 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.791 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.792 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.793 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.794 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.795 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.796 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.797 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.798 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.799 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.802 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.803 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.805 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.806 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.807 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.808 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.809 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.810 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.811 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.812 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.813 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.814 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.815 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.816 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.817 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.818 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.819 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.820 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.821 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.822 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.823 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-09-12T03:37:18.824 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.825 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.826 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.827 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.828 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.829 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.830 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.831 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.832 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-09-12T03:37:18.833 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-09-12T03:37:18.834 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.835 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.837 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.838 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.839 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-09-12T03:37:18.840 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.841 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.842 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.843 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.844 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.845 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.846 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.847 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.848 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-09-12T03:37:18.849 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-09-12T03:37:18.850 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-09-12T03:37:18.851 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.852 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.853 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.854 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.855 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.856 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.857 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.858 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.859 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.860 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.861 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.863 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-09-12T03:37:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-09-12T03:37:18.865 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.866 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.867 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.868 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.869 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.870 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.871 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.872 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.873 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.874 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.875 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.876 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.880 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.881 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.883 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.884 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.885 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.886 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.887 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.888 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.889 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.890 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.891 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.892 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.893 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.893 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.893 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.896 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.896 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.896 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.896 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.896 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.896 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.897 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.898 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.899 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.900 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.903 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-09-12T03:37:18.904 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.905 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.906 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.907 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.909 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-09-12T03:37:18.910 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.911 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.913 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.914 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.915 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.916 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.917 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.918 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.919 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.920 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.921 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.922 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.923 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.924 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.925 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.926 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.927 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.928 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.929 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.930 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.931 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.932 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.933 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.934 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.935 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.936 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.937 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.938 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.939 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.940 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.941 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:37:18.942 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.944 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.945 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.946 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.947 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.948 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.949 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.950 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.951 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.952 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.953 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.954 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.955 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.956 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.958 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.960 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.961 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.962 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.963 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.964 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.965 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.967 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.968 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.969 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.970 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.971 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.972 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.976 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.977 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.978 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.979 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.980 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.981 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.982 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.983 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.984 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.985 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-09-12T03:37:18.986 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:18.987 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.988 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.991 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.992 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.994 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.995 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.997 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.998 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:18.999 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.000 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-09-12T03:37:19.001 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.003 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.004 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.005 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.006 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.007 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.008 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.009 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.010 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.012 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.013 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.015 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.016 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.017 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.018 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.019 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.020 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.021 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.022 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.023 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-09-12T03:37:19.024 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.025 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.026 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.027 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.028 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.029 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.030 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.032 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.033 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.034 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.035 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.036 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.037 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.038 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.039 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.040 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.041 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-09-12T03:37:19.042 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-09-12T03:37:19.043 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-09-12T03:37:19.044 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.045 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.046 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.047 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.048 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.050 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.051 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.052 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.053 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.054 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.068 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:19.069 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.070 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.071 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.072 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.073 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.074 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.075 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.076 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.077 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.078 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.079 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "dashboard", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "restful", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "snap_schedule" 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.081 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.082 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.083 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.084 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.085 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.086 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.087 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.088 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.089 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.090 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.091 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.092 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.095 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.096 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.097 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:19.098 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-09-12T03:37:19.099 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:37:19.100 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-09-12T03:37:19.101 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-09-12T03:37:19.102 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-09-12T03:37:19.103 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.104 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.105 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.107 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.108 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.109 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.110 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-09-12T03:37:19.111 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:19.112 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-09-12T03:37:19.113 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:37:19.114 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.115 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.116 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.117 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.118 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.119 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.120 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.121 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.122 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.124 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.126 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.127 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.128 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.129 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.130 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.131 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.132 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.133 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.134 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.135 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.136 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.137 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.138 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.139 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.140 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.141 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.142 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.143 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.144 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.145 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.146 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.147 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.148 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.149 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.150 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.151 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.152 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.153 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.154 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.156 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.157 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.158 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.159 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.160 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.161 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.162 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.163 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-09-12T03:37:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.166 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.168 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.169 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.170 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.171 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.172 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.174 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:37:19.175 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.176 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.177 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.178 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.179 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.180 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.181 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.182 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.183 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.184 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.185 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.186 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.187 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.188 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.189 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.190 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.191 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.193 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.194 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.195 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.196 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.197 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.198 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.199 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.200 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.201 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.202 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.203 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.204 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.205 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.206 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:37:19.207 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:37:19.208 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:37:19.209 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.210 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.211 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.212 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.213 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.214 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.216 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.217 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.218 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.219 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.220 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.221 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.222 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.223 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.224 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.225 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.227 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.228 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.229 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.230 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.231 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.232 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.233 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.234 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.235 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.236 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.237 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.238 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.240 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.241 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.243 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-09-12T03:37:19.244 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-09-12T03:37:19.245 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.246 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-09-12T03:37:19.247 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-09-12T03:37:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.249 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.251 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.252 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.253 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.254 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.255 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.256 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.257 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.258 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.259 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-09-12T03:37:19.260 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.261 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.262 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.263 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.264 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-09-12T03:37:19.265 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.267 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.268 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.269 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.270 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.271 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-09-12T03:37:19.272 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-09-12T03:37:19.273 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:37:19.274 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.275 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.276 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.277 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.278 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.279 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.280 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.282 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.283 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:37:19.284 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-09-12T03:37:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:37:19.286 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.287 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:37:19.288 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:37:19.289 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.290 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.292 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.293 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.294 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.295 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.296 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.297 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.298 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.299 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.300 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-09-12T03:37:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.304 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.306 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.307 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.308 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.309 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.310 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.311 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.312 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.313 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.314 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.315 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.316 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.317 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.318 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-09-12T03:37:19.319 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.320 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.321 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.322 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-09-12T03:37:19.323 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.324 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.325 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.326 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.327 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.328 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.329 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.330 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.331 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.332 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.333 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.334 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.336 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.337 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.338 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.339 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.340 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.341 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.342 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.343 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.344 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.345 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.346 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.348 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.349 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.350 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.351 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.352 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.354 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.355 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.356 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.357 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.358 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.359 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.360 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.362 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.363 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.364 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.365 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-09-12T03:37:19.366 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.367 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.368 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.369 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.370 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.371 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.373 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.374 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.375 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.376 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.377 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.378 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.379 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.380 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.381 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.382 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.383 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-09-12T03:37:19.384 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:37:19.385 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-09-12T03:37:19.386 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:19.387 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:19.388 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-09-12T03:37:19.389 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-09-12T03:37:19.390 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.391 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.392 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.393 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.395 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.396 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.397 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-09-12T03:37:19.398 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:37:19.399 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.400 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.401 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.402 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.403 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.404 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.405 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.406 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.407 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.410 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.411 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.412 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.413 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.414 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.416 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.417 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.418 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.419 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-09-12T03:37:19.422 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.423 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.424 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.425 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.426 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.427 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.428 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.429 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.430 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.431 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.432 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.433 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.434 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.436 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.437 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.438 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.439 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.440 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.441 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.442 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.443 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.444 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.445 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.446 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.447 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.448 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.449 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.450 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.451 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.452 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.453 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-09-12T03:37:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.455 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.456 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.457 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.458 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-09-12T03:37:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.461 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.462 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.463 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.464 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.465 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.466 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.467 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.468 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.469 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.470 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.471 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.472 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.473 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.474 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:37:19.475 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.476 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.477 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-09-12T03:37:19.478 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.479 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.480 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.481 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.482 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.483 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.485 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.486 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.488 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.489 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.491 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.492 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.493 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.494 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.495 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.496 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.497 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.498 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.499 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.500 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.501 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.502 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:37:19.503 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.504 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.505 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-09-12T03:37:19.505 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.508 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.509 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.510 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.511 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.512 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.513 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-09-12T03:37:19.514 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-09-12T03:37:19.515 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-09-12T03:37:19.517 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-09-12T03:37:19.518 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.519 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "dashboard": "https://172.21.15.50:8443/" 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "always_on_modules": { 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "octopus": [ 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-09-12T03:37:19.520 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "pacific": [ 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.521 INFO:teuthology.orchestra.run.smithi050.stdout: "quincy": [ 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "reef": [ 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-09-12T03:37:19.522 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "squid": [ 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-09-12T03:37:19.523 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "last_failure_osd_epoch": 95, 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "active_clients": [ 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.524 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2277067334 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "libcephsqlite", 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 33399773 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.525 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 319954119 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-09-12T03:37:19.526 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 2673200263 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-09-12T03:37:19.527 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 543690056 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1294246264 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:19.528 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-12T03:37:19.529 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:37:19.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.129+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd0c00a2d80 msgr2=0x7fd0c00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:19.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.129+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd0c00a2d80 0x7fd0c00a5240 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000c00 tx=0x7fd0c8005b50 comp rx=0 tx=0).stop 2024-09-12T03:37:19.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.129+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 msgr2=0x7fd0dc19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:19.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.129+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc19a3a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00ea60 tx=0x7fd0cc00ef20 comp rx=0 tx=0).stop 2024-09-12T03:37:19.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 shutdown_connections 2024-09-12T03:37:19.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc1078f0 0x7fd0dc19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:19.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0dc069d00 0x7fd0dc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:19.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd0c00a2d80 0x7fd0c00a5240 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:19.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 --2- 172.21.15.50:0/4081246601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0dc069380 0x7fd0dc19a3a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:19.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 >> 172.21.15.50:0/4081246601 conn(0x7fd0dc076550 msgr2=0x7fd0dc10cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:19.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 shutdown_connections 2024-09-12T03:37:19.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:18.130+0000 7fd0dbe00640 1 -- 172.21.15.50:0/4081246601 wait complete. 2024-09-12T03:37:19.531 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:17 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3765733234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.531 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:17 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1612607205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.531 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:17 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/145267479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.531 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:18 smithi082 ceph-mon[35046]: pgmap v314: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 135 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:18 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4081246601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:17 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3765733234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:17 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1612607205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:17 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/145267479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:18 smithi142 ceph-mon[35329]: pgmap v314: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 135 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:18 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4081246601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:17 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3765733234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:17 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1612607205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.532 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:17 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/145267479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:37:19.539 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:37:19.539 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:37:19.540 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:18 smithi050 ceph-mon[30378]: pgmap v314: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 135 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-12T03:37:19.540 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:18 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4081246601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:37:19.930 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:20.593 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:37:20.593 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:20.593 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 222, 2024-09-12T03:37:20.593 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 211, 2024-09-12T03:37:20.593 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1580816, 2024-09-12T03:37:20.593 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 129344, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.055538076, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022237629, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.044954235, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 210, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.018032208, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000085867 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.594 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1580816, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 129344 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 161, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 185472, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:20.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.020874327, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.013801378, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003232886, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 158, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.025442593, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000161029 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:20.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 185472 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 4, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 4, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 448, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.003849683, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000072472, 2024-09-12T03:37:20.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003683336, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.006284486, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002094828 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:37:20.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:20.599 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:20.600 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:20.601 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148977, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 30, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:20.602 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-09-12T03:37:20.603 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 224, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1549520, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 224, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 224, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1549520, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:20.604 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:37:21.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:20 smithi050 ceph-mon[30378]: pgmap v315: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-09-12T03:37:21.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:20 smithi142 ceph-mon[35329]: pgmap v315: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-09-12T03:37:21.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:20 smithi082 ceph-mon[35046]: pgmap v315: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-09-12T03:37:21.482 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T03:37:21.482 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T03:37:21.482 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-12T03:37:21.482 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-12T03:37:21.482 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-12T03:37:21.482 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-12T03:37:21.482 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:37:21.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:37:21.870 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:22.596 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:37:22.596 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:22.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 228, 2024-09-12T03:37:22.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 217, 2024-09-12T03:37:22.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1582448, 2024-09-12T03:37:22.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 133184, 2024-09-12T03:37:22.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:22.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-09-12T03:37:22.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.050357942, 2024-09-12T03:37:22.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.016249787, 2024-09-12T03:37:22.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.045705082, 2024-09-12T03:37:22.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:22.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:22.599 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 216, 2024-09-12T03:37:22.599 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.015656977, 2024-09-12T03:37:22.599 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000072486 2024-09-12T03:37:22.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.599 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:22.599 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:22.600 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:22.600 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:22.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1582448, 2024-09-12T03:37:22.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 133184 2024-09-12T03:37:22.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.601 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:22.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-09-12T03:37:22.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 166, 2024-09-12T03:37:22.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:37:22.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 186352, 2024-09-12T03:37:22.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:22.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:22.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.021471251, 2024-09-12T03:37:22.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.014854480, 2024-09-12T03:37:22.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003107225, 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 163, 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.020673136, 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000126829 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:22.603 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 186352 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 4, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 4, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 448, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:22.604 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.003570931, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000065969, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003405686, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.006504854, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002168284 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:22.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:22.606 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:22.607 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:22.608 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:22.609 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148714, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 30, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:22.610 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-09-12T03:37:22.611 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 230, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1550300, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 230, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-09-12T03:37:22.612 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 230, 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1550300, 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:22.613 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:37:23.258 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:22 smithi050 ceph-mon[30378]: pgmap v316: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 293 B/s wr, 9 op/s 2024-09-12T03:37:23.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:22 smithi142 ceph-mon[35329]: pgmap v316: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 293 B/s wr, 9 op/s 2024-09-12T03:37:23.295 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:37:23.295 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:37:23.295 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T03:37:23.295 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:37:23.295 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:37:23.295 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T03:37:23.295 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:37:23.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:37:23.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:22 smithi082 ceph-mon[35046]: pgmap v316: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 293 B/s wr, 9 op/s 2024-09-12T03:37:23.674 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:24.253 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 224, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 213, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1581360, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 130624, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.055398284, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022263835, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.044320469, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:24.254 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 212, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.020295560, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000095733 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1581360, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 130624 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.255 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 163, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 185824, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.019716649, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.013537585, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003004266, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 160, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.020505363, 2024-09-12T03:37:24.256 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000128158 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 185824 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 4, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 4, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:37:24.257 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 448, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.003473705, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000067370, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.003312404, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.005923168, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001974389 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:24.258 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-09-12T03:37:24.259 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:24.260 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:24.261 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148994, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 31, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-09-12T03:37:24.262 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-09-12T03:37:24.263 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 226, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1549780, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 226, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-09-12T03:37:24.264 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 226, 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1549780, 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:24.265 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:37:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:24 smithi050 ceph-mon[30378]: pgmap v317: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:37:24.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:24 smithi050 ceph-mon[30378]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:37:24.996 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T03:37:24.997 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T03:37:24.997 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-12T03:37:24.997 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:37:24.997 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:37:24.997 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-12T03:37:24.997 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:37:24.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:37:25.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:24 smithi142 ceph-mon[35329]: pgmap v317: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:37:25.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:24 smithi142 ceph-mon[35329]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:37:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:24 smithi082 ceph-mon[35046]: pgmap v317: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:37:25.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:24 smithi082 ceph-mon[35046]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:37:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-09-12T03:37:25.944 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.003164882, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000578353, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:25.945 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 230, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 219, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1582992, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 134464, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.053400556, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022727214, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.045888445, 2024-09-12T03:37:25.946 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 218, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.014907730, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000068384 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1582992, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 134464 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.947 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 6, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 169, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3312, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 186976, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.024829538, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.015860621, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.005528422, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 167, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.036347422, 2024-09-12T03:37:25.948 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000217649 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 186976 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-09-12T03:37:25.949 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:25.950 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:25.951 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148731, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 31, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:25.952 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-09-12T03:37:25.953 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 232, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1550560, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 232, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-09-12T03:37:25.954 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 232, 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1550560, 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-12T03:37:25.955 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-12T03:37:26.674 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:37:26.674 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:37:26.674 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T03:37:26.674 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:37:26.674 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:37:26.674 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T03:37:26.674 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:37:26.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:37:26.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:26 smithi050 ceph-mon[30378]: pgmap v319: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-12T03:37:26.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:26 smithi050 ceph-mon[30378]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:37:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:37:27.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:26 smithi142 ceph-mon[35329]: pgmap v319: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-12T03:37:27.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:26 smithi142 ceph-mon[35329]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:37:27.053 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:26 smithi082 ceph-mon[35046]: pgmap v319: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-12T03:37:27.053 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:26 smithi082 ceph-mon[35046]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 43741, 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 39365, 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 14153791, 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 96720586, 2024-09-12T03:37:27.496 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 66, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 2.559471804, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.869059737, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.884002494, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.765398242, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 39367, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.250482370, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000082568 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:27.497 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 12638, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002656272, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000210 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 45536, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 1584 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 25617, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 24545, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 9026692, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 54142785, 2024-09-12T03:37:27.498 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 1.555923215, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.504179507, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.572445701, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.462119125, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 24547, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.634518292, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000148063 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:27.499 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8105, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001271872, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000156 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1691888, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 454480 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 57965, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 109604, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 17886174, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 62100476, 2024-09-12T03:37:27.500 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 27, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 8, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 3.715971162, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 1.597354372, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 1.941444664, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.533915100, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 109595, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 8.454058861, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000077139 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 67461, 2024-09-12T03:37:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.020313604, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000301 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 392832, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 259936 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-09-12T03:37:27.502 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 90096, 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 22.603473822, 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000250882 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3681, 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.520109394, 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000141295 2024-09-12T03:37:27.503 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 19054, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 18771, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 18771, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 66.107200055, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003521772 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 283, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 1, 2024-09-12T03:37:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 763, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 22, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 130, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 5, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 125, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 129, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 6574, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 2, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 3, 2024-09-12T03:37:27.505 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 21023, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 20028, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 228, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 1, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 131, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 5966, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 9, 2024-09-12T03:37:27.506 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 320, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 4, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 234, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 6, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:37:27.507 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 16, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 8714, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 17737, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 7863, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 15011, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 12752, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 547, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 11610 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.508 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 102, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 3, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 2010, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 2004, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 3, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-09-12T03:37:27.509 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 131, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 22, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.510 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 42629, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 42414, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 42414, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 215, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 22, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 81, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 80, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 80, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 1, 2024-09-12T03:37:27.511 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 1, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 1, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 139021626, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 139790567, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 25126, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 56.024364184, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002229736 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.512 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 62, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 7620, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 7558, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 21, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 337, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 316, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 130, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 8770, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 8640, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 6504, 2024-09-12T03:37:27.513 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 6502, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 96864, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231712 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 37579, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 1589, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 19054, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 159, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 3234, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6366, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 19.833753026, 2024-09-12T03:37:27.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003115575 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 301, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.593073325, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.005292602 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.007044206, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002348068 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 582, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.555923882, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000955195 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.516 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-09-12T03:37:27.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001748631, 2024-09-12T03:37:27.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000174863 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 69, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055740861, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000807838 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-09-12T03:37:27.519 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.120234250, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.012023425 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 394, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.072618613, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000184311 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 147, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.626909087, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.004264687 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.520 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 91, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.241463234, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002653442 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 15, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.076308736, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.005087249 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 4.193388762, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 1.048347190 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8815, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 34.918577896, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003961268 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8, 2024-09-12T03:37:27.523 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003182691, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000397836 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.022596977, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002824622 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.524 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.004369317, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000728219 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1942, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.780266561, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001946584 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-09-12T03:37:27.525 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 6, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 4, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 315, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 157, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 590, 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-09-12T03:37:27.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 360, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 360, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:27.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:27.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 316363, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 173, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 1232, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 14, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:27.529 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9704, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 196, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 310210, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2816, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.530 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 77125, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 146103611, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 31, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 77125, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 77125, 2024-09-12T03:37:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3050.100400778, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.039547493 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 77125, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 92714 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 77125, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 63, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 77062, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 512, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 9403, 2024-09-12T03:37:27.532 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 58, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 25184, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 164, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 6, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 4642, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 14170, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:37:27.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 36717, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-09-12T03:37:27.534 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 104, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-09-12T03:37:27.535 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 20, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 1733, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 6, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 118 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-09-12T03:37:27.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 56, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 10, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 4050, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 2004, 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-09-12T03:37:27.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 127319, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 30112842, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 127319, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 127319, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 30112842, 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:27.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:27.539 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 77125, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 146324702, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 77125, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 146324702, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:27.540 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 77125, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 77125, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 77125, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 77125, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2004, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 221236, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2004, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 58, 2024-09-12T03:37:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 221236, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x56143cf3c960": { 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 42629, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 135596263, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 42629, 2024-09-12T03:37:27.543 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 25126, 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 135596263, 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:27.544 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-09-12T03:37:28.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:27 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:37:28.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:27 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:28.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:27 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:27 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:37:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:27 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:28.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:27 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:27 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:37:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:27 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:28.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:27 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:28.377 INFO:tasks.check_counter:Daemon mds.b mds.exported=131 2024-09-12T03:37:28.377 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2024-09-12T03:37:28.377 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-12T03:37:28.377 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-12T03:37:28.377 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-12T03:37:28.377 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-09-12T03:37:28.377 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:37:28.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:37:29.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:28 smithi142 ceph-mon[35329]: pgmap v320: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:37:29.106 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:28 smithi082 ceph-mon[35046]: pgmap v320: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:37:29.145 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:37:29.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:28 smithi050 ceph-mon[30378]: pgmap v320: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 261, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 360, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1666720, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 447264, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.046973138, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.018555477, 2024-09-12T03:37:29.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.041788303, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 359, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.034757663, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000096818 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1666720, 2024-09-12T03:37:29.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 447264 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 974, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1012, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 12582598, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 435715, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.078514872, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.022571255, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.027531077, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.021805407, 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:29.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1010, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.043380696, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000042951 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 127568 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:29.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1154, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1156, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4548057, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 367036, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.114801144, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.035581682, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.032128229, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.036850896, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1154, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.044877172, 2024-09-12T03:37:29.633 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000038888 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-09-12T03:37:29.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3576, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.091061043, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000025464 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-09-12T03:37:29.636 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 2307, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 2307, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 2307, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-09-12T03:37:29.637 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 1, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-09-12T03:37:29.638 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:37:29.639 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-09-12T03:37:29.640 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-09-12T03:37:29.641 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-09-12T03:37:29.642 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 8943, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 12, 2024-09-12T03:37:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 20719746, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 20716239, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 8943 2024-09-12T03:37:29.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 99, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 4689, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 4590, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 16, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 104, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 88, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 2307, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 6429, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 4122, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-09-12T03:37:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 60912, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215328 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.646 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.648 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.649 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.650 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.651 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.652 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.655 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.656 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-09-12T03:37:29.657 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 3320, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:29.658 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:29.659 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:29.660 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 202684, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 158, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:29.661 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9704, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 196, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 1897891, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 14724, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:29.662 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2123, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2123, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2123, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.914960288, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000902006 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.663 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2123, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2124 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2123, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2123, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1339, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 782, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-09-12T03:37:29.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:37:29.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-09-12T03:37:29.666 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 104, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 12, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 6, 2024-09-12T03:37:29.667 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 6, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.668 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2385, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 18566482, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2385, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2385, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 18566482, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.670 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2123, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 16527029, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2123, 2024-09-12T03:37:29.671 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 16527029, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2123, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2123, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2123, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2123, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.673 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:29.674 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x563ae3f4ab40": { 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-09-12T03:37:29.675 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.676 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:29.677 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.045 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:29 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.104 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.105 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:29 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:29 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:30.384 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T03:37:30.384 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T03:37:30.384 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-12T03:37:30.384 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:37:30.384 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:37:30.384 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-12T03:37:30.384 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:37:30.384 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:37:30.648 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:37:31.079 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-12T03:37:31.079 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: Unable to set osd_memory_target on smithi142 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: pgmap v321: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.080 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:30 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: Unable to set osd_memory_target on smithi142 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: pgmap v321: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:30 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.204 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1958, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1960, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 17337547, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 622651, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.146067861, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.046042593, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.043705373, 2024-09-12T03:37:31.205 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.045117165, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1958, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.053289727, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000027216 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.206 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 486, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 459, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 8829955, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 207230, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.063314640, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.020616676, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.050853033, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004802181, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 459, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.024087528, 2024-09-12T03:37:31.207 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000052478 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1669008, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 135936 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 23, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 186, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4200052, 2024-09-12T03:37:31.208 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 445734, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.013302362, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.006804428, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.004018518, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000502454, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 183, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.017030886, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000093064 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:31.209 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 439056 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3687, 2024-09-12T03:37:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.080311852, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000021782 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-09-12T03:37:31.211 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 14659, 2024-09-12T03:37:31.212 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 560, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 14099, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 13859, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 1, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:37:31.213 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-09-12T03:37:31.214 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:37:31.215 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-09-12T03:37:31.216 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-09-12T03:37:31.217 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 9517, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-09-12T03:37:31.218 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 14, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 32263781, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 32259284, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.219 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 9517 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 6636, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 9918, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 3282, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 269, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 287, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 18, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 14659, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 15004, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 345, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-09-12T03:37:31.220 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 83476, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215328 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.221 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-09-12T03:37:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-09-12T03:37:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 5, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 3, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 4, 2024-09-12T03:37:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 3323, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 447261, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 6768, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9704, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 196, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 26108676, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 220693, 2024-09-12T03:37:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2203, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2203, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2203, 2024-09-12T03:37:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.114976866, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000506117 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2203, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2204 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2203, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2203, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-09-12T03:37:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1400, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 801, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-09-12T03:37:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-09-12T03:37:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-09-12T03:37:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 104, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 12, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 6, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 6, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:37:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-09-12T03:37:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2463, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 30129745, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2463, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2463, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 30129745, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2203, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 28077722, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2203, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 28077722, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2203, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2203, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2203, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2203, 2024-09-12T03:37:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55bd428f6b40": { 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-09-12T03:37:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:31.251 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-09-12T03:37:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-09-12T03:37:31.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: Unable to set osd_memory_target on smithi142 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: pgmap v321: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:30 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.968 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-09-12T03:37:31.968 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: Unable to set osd_memory_target on smithi050 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:37:31.968 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.968 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: Adjusting osd_memory_target on smithi082 to 176.0M 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: Unable to set osd_memory_target on smithi082 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:37:31.969 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:31 smithi082 ceph-mon[35046]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:31.995 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T03:37:31.995 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T03:37:31.995 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-12T03:37:31.995 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:37:31.995 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:37:31.995 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-12T03:37:31.995 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:37:31.995 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:37:32.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: Unable to set osd_memory_target on smithi050 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: Adjusting osd_memory_target on smithi082 to 176.0M 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: Unable to set osd_memory_target on smithi082 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:37:32.165 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:31 smithi050 ceph-mon[30378]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi050 to 177.5M 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: Unable to set osd_memory_target on smithi050 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:37:32.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: Adjusting osd_memory_target on smithi082 to 176.0M 2024-09-12T03:37:32.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: Unable to set osd_memory_target on smithi082 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T03:37:32.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:37:32.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:37:32.295 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:31 smithi142 ceph-mon[35329]: from='mgr.25157 ' entity='mgr.x' 2024-09-12T03:37:32.764 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-09-12T03:37:32.764 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:32.764 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-09-12T03:37:32.764 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-09-12T03:37:32.764 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:37:32.764 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000988322, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000193446, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:32.765 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 232, 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 221, 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1583536, 2024-09-12T03:37:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 135744, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.040825133, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.013476592, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.041721887, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 220, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.012090701, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000054957 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1583536, 2024-09-12T03:37:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 135744 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 6, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 172, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3312, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 187504, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.012383300, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.006877666, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.003531638, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 170, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.030310724, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000178298 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 187504 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148754, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 31, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-09-12T03:37:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-09-12T03:37:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 234, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1550820, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 234, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 234, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1550820, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-09-12T03:37:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-09-12T03:37:32.778 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-09-12T03:37:33.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:33 smithi142 ceph-mon[35329]: pgmap v322: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:33.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:33 smithi082 ceph-mon[35046]: pgmap v322: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:33.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:33 smithi050 ceph-mon[30378]: pgmap v322: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:33.420 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T03:37:33.420 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T03:37:33.420 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-12T03:37:33.420 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:37:33.420 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:37:33.420 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-12T03:37:33.420 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:37:33.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:37:33.678 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1871, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1875, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 793643, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 1189645, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 15, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.144210060, 2024-09-12T03:37:34.297 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.046559288, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.035876317, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.048287361, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1871, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.174611474, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000093325 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000334, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000083 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.298 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 3957, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 5867, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1177584, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 11954036, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 11, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.355975106, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.162560487, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.137710041, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.048717124, 2024-09-12T03:37:34.299 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5866, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.335166003, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000057137 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6145, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001099481, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000178 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2160, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 442112 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.300 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 13772, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 21975, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 7902719, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 17428082, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 7, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.795228790, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.359321437, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.525482814, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.042532836, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 21975, 2024-09-12T03:37:34.301 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.283203902, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000058393 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 17431, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.004303993, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000246 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1679840, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 143344 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-12T03:37:34.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 557, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.872107109, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005156386 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3865, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.619067822, 2024-09-12T03:37:34.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000160172 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 5578, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 5578, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5578, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 9.439258553, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001692229 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 1, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 11, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 169, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 2, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 2, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 165, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 165, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 4116, 2024-09-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 1, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 1, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 3, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 11164, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 7526, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 30, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 68, 2024-09-12T03:37:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 4078, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 68, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 1820, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:37:34.307 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 79, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 2948, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 2484, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 2281, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 99, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 4480 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.308 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 10, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 68, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 8, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 2153, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 2145, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-12T03:37:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 68, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-12T03:37:34.310 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 8943, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 8943, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 12, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 12, 2024-09-12T03:37:34.311 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 20719746, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 318, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.550755969, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.011165899 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-09-12T03:37:34.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 108, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 4700, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 4592, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 25, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 104, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 79, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 169, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 6586, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 6417, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 1, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 3774, 2024-09-12T03:37:34.313 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 3773, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 78096, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 223520 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 8611, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 7, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 5578, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 13, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 2692, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 583, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.790907028, 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001356615 2024-09-12T03:37:34.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000372680, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000372680 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2662, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5.064735851, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001902605 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-09-12T03:37:34.316 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000084095, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000084095 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.317 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.004613106, 2024-09-12T03:37:34.318 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001153276 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 27, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.006761198, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000250414 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 147, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.118870939, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000808645 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.320 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.321 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.145753158, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.018219144 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-12T03:37:34.323 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.324 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2145, 2024-09-12T03:37:34.325 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.307160498, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001541799 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-09-12T03:37:34.326 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 3003, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 3003, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 615, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:34.327 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:34.328 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:34.329 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 357622, 2024-09-12T03:37:34.330 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 199, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 528, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 6, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9704, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 196, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:34.331 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 437995, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 6586, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:34.332 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 4823, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 17008772, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 33, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 4823, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4823, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 43.192143019, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008955451 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.333 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4823, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4919 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 4823, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 42, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 4781, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 40, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 358, 2024-09-12T03:37:34.334 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 32, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 4305, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-12T03:37:34.335 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-12T03:37:34.336 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-12T03:37:34.337 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-12T03:37:34.337 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-12T03:37:34.337 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-12T03:37:34.337 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-12T03:37:34.337 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-12T03:37:34.337 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 104, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 12, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 122, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 3, 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 28 2024-09-12T03:37:34.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.339 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 16, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 8, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 4290, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 2145, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:34.340 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 19596, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 8230336, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 19596, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 19596, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 8230336, 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:34.341 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:34.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:34.343 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:34.343 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 4823, 2024-09-12T03:37:34.343 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 17316146, 2024-09-12T03:37:34.343 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4823, 2024-09-12T03:37:34.343 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 17316146, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:34.344 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 4823, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 4823, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4823, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 4823, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2145, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 307413, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2145, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 40, 2024-09-12T03:37:34.346 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 307413, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5622fc68e960": { 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 8943, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 16525442, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:34.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 8943, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 318, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 16525442, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:34.348 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T03:37:35.195 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:35 smithi142 ceph-mon[35329]: pgmap v323: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:35.223 INFO:tasks.check_counter:Daemon mds.c mds.exported=68 2024-09-12T03:37:35.223 INFO:tasks.check_counter:Daemon mds.c mds.imported=68 2024-09-12T03:37:35.223 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-12T03:37:35.223 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:37:35.223 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:37:35.223 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-12T03:37:35.223 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:37:35.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:37:35.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:35 smithi082 ceph-mon[35046]: pgmap v323: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:35.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:35 smithi050 ceph-mon[30378]: pgmap v323: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:37:35.476 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:37:35.994 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 252, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 253, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 29426825, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 79645, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.034841108, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.005143166, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.020570130, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.005200367, 2024-09-12T03:37:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 252, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.018931765, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000075126 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2656, 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:37:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1275, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1383, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 66395027, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 683433, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.143199622, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.036743259, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.060502408, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.036131007, 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1381, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.099439247, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000072005 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 278240 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 588, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 579, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 22710305, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 223989, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.060831650, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.019873529, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.033600274, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.008973821, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 578, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.038854356, 2024-09-12T03:37:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000067222 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 237888, 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 89792 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3272, 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.093323600, 2024-09-12T03:37:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000028521 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-12T03:37:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1910, 2024-09-12T03:37:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 1205, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 705, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 189, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 2, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:37:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-09-12T03:37:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:37:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-12T03:37:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:37:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 34625, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-12T03:37:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 25, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 139021626, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 139790567, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 139790431, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 34625 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 90, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 4882, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 4792, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 19, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 285, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 266, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1910, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 5294, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 3384, 2024-09-12T03:37:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 60232, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 215328 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-12T03:37:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-12T03:37:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-12T03:37:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-12T03:37:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 0, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1978, 2024-09-12T03:37:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 233054, 2024-09-12T03:37:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 159, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9704, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 196, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 1613786, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 9960, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-12T03:37:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 1945, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 26, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 4, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 1945, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1945, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.729965616, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000889442 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1945, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1946 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 1945, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 1945, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1065, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 877, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-12T03:37:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-12T03:37:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-12T03:37:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-12T03:37:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 104, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 8, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 10, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 10, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-09-12T03:37:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-12T03:37:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2112, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 118330947, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2112, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2112, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 118330947, 2024-09-12T03:37:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 1945, 2024-09-12T03:37:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 117737886, 2024-09-12T03:37:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1945, 2024-09-12T03:37:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 117737886, 2024-09-12T03:37:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 1945, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1945, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1945, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1945, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:37:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55b1e1650b40": { 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:36.047 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T03:37:36.801 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:37:36.801 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:37:36.801 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-12T03:37:36.801 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:37:36.801 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:37:36.802 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-12T03:37:36.802 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:37:36.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:37:37.049 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:37:37.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:37 smithi142 ceph-mon[35329]: pgmap v324: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:37:37.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:37 smithi082 ceph-mon[35046]: pgmap v324: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:37:37.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:37 smithi050 ceph-mon[30378]: pgmap v324: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:37:37.568 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2777, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2908, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 732921, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 18364617, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 16, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.240664155, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.095152162, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.051968925, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.073664050, 2024-09-12T03:37:37.569 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2905, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.313253404, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000107832 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000003286, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000142 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 317680 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.570 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 14610, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 22178, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 8414809, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 16672199, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.886066417, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.413678935, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.545741607, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.037174668, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22177, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.759031738, 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000079317 2024-09-12T03:37:37.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 18346, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.004206282, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000229 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1679920, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 141264 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 6967, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 11319, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 6247573, 2024-09-12T03:37:37.572 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 12340977, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 11, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.536094443, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.209049577, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.250924700, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.076395497, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 11318, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.907166846, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000080152 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:37.573 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 12055, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.002675437, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000221 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 7216, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 140720 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-12T03:37:37.574 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 664, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.598721818, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006925785 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5493, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.749893145, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000136517 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.575 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 6722, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 6716, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6716, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 10.110236794, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001505395 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 6, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 14, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 75, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 8, 2024-09-12T03:37:37.576 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 11319, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 577, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 10657, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 85, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 10493, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 426, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 3, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 3, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 3, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 10729, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 9641, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 2, 2024-09-12T03:37:37.577 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 25, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 9, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 320, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 131, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 8224, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:37:37.578 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 15, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 153, 2024-09-12T03:37:37.579 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 3542, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 1972, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 1746, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 56, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 6252 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-12T03:37:37.580 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 160, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 50, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 3116, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 3066, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-12T03:37:37.581 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 9, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 8, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 1, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 9517, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-12T03:37:37.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 9517, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 14, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 14, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 32263781, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-12T03:37:37.583 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 330, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.850406810, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008637596 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 6690, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 10008, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 3318, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 277, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 290, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 13, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 11319, 2024-09-12T03:37:37.584 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 15128, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 3809, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 3, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 5606, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 5603, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 101652, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 223520 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 8578, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 1702, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 6722, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 32, 2024-09-12T03:37:37.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 3981, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 258, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.356499734, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001381781 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 7, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005096871, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000728124 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3041, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.430682828, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000470464 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.587 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000126511, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000126511 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.588 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.590 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000312885, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000156442 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 290, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.345021802, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001189730 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-12T03:37:37.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.006835369, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002278456 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 50, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.389399750, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.047787995 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.595 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3064, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5.576261044, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001819928 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 5, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 5094, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 2547, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 600, 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-12T03:37:37.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:37.598 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:37.599 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 942209, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 6291, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 880, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 10, 2024-09-12T03:37:37.600 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9704, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 196, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 24071460, 2024-09-12T03:37:37.601 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 234543, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 6819, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 29249825, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 29, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 6819, 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-12T03:37:37.602 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6819, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 69.049630979, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.010126064 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6819, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 6996 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 6819, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 88, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 6731, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-12T03:37:37.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 53, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 383, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 42, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 6160, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 6, 2024-09-12T03:37:37.604 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-12T03:37:37.605 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 104, 2024-09-12T03:37:37.606 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 12, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 242, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 49, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 31 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-12T03:37:37.607 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 19, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 9, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 6134, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 3066, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.608 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 24350, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 13328526, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 24350, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 24350, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 13328526, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.609 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 6819, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 29707706, 2024-09-12T03:37:37.610 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 6819, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 29707706, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:37.611 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 6819, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 6819, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 6819, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 6819, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:37.612 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 3066, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 458626, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 3066, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 53, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 458626, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55e1b1a28960": { 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 9517, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 28069477, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 9517, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 330, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 28069477, 2024-09-12T03:37:37.614 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:37.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:37.615 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:37.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:37.615 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.615 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:37.615 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T03:37:38.386 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-09-12T03:37:38.386 INFO:tasks.check_counter:Daemon mds.i mds.imported=131 2024-09-12T03:37:38.386 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-12T03:37:38.386 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:37:38.386 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:37:38.386 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=8 2024-09-12T03:37:38.386 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:37:38.387 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:37:38.633 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:37:39.170 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T03:37:39.170 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 235, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 224, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1584352, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 137664, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 0, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.041146016, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.014270993, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.039761022, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 223, 2024-09-12T03:37:39.171 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.011771682, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000052787 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1584352, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 137664 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 6, 2024-09-12T03:37:39.172 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 173, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 3312, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 187680, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.012234735, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.006810065, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.003379984, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 171, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.029670644, 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000173512 2024-09-12T03:37:39.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 187680 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 0, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 0, 2024-09-12T03:37:39.174 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 2, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.000235421, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.000168805, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:39.175 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 0, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-12T03:37:39.176 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:37:39.177 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:37:39.178 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 148731, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 31, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-12T03:37:39.179 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 928, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 2, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-12T03:37:39.180 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 237, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1551210, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 237, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-12T03:37:39.181 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 237, 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1551210, 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T03:37:39.182 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T03:37:39.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:39 smithi142 ceph-mon[35329]: pgmap v325: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:37:39.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:39 smithi082 ceph-mon[35046]: pgmap v325: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:37:39.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:39 smithi050 ceph-mon[30378]: pgmap v325: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:37:39.979 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-12T03:37:39.980 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-12T03:37:39.980 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-12T03:37:39.980 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-12T03:37:39.980 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-12T03:37:39.980 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-12T03:37:39.980 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:37:39.990 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:37:39.991 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:37:39.991 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:37:40.032 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-12T03:37:40.032 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:37:40.112 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:37:40.534 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 -- 172.21.15.50:0/513178302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc610109120 msgr2=0x7fc610110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 --2- 172.21.15.50:0/513178302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc610109120 0x7fc610110ca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc608009ca0 tx=0x7fc60802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 -- 172.21.15.50:0/513178302 shutdown_connections 2024-09-12T03:37:41.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 --2- 172.21.15.50:0/513178302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc610109120 0x7fc610110ca0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 --2- 172.21.15.50:0/513178302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6101049a0 0x7fc6101089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 --2- 172.21.15.50:0/513178302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610103ff0 0x7fc6101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 -- 172.21.15.50:0/513178302 >> 172.21.15.50:0/513178302 conn(0x7fc6100ffa90 msgr2=0x7fc610101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 -- 172.21.15.50:0/513178302 shutdown_connections 2024-09-12T03:37:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.068+0000 7fc616600640 1 -- 172.21.15.50:0/513178302 wait complete. 2024-09-12T03:37:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.069+0000 7fc616600640 1 Processor -- start 2024-09-12T03:37:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.069+0000 7fc616600640 1 -- start start 2024-09-12T03:37:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc616600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc610103ff0 0x7fc61019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:41.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc616600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6101049a0 0x7fc61019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc616600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610109120 0x7fc61019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc616600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101141a0 con 0x7fc6101049a0 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc616600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc610114020 con 0x7fc610103ff0 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc616600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc610114320 con 0x7fc610109120 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc60f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc610103ff0 0x7fc61019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc606a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6101049a0 0x7fc61019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc60fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610109120 0x7fc61019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc606a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6101049a0 0x7fc61019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34054/0 (socket says 172.21.15.50:34054) 2024-09-12T03:37:41.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc60fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610109120 0x7fc61019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:41038/0 (socket says 172.21.15.50:41038) 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.070+0000 7fc606a00640 1 -- 172.21.15.50:0/866167941 learned_addr learned my addr 172.21.15.50:0/866167941 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60f400640 1 -- 172.21.15.50:0/866167941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610109120 msgr2=0x7fc61019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60f400640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610109120 0x7fc61019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60f400640 1 -- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6101049a0 msgr2=0x7fc61019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60f400640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6101049a0 0x7fc61019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60f400640 1 -- 172.21.15.50:0/866167941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61019f590 con 0x7fc610103ff0 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60f400640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc610103ff0 0x7fc61019a5e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e9f0 tx=0x7fc60000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60000ccc0 con 0x7fc610103ff0 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61019f880 con 0x7fc610103ff0 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc600005020 con 0x7fc610103ff0 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.071+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000049e0 con 0x7fc610103ff0 2024-09-12T03:37:41.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.072+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6101a7160 con 0x7fc610103ff0 2024-09-12T03:37:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.073+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6101043d0 con 0x7fc610103ff0 2024-09-12T03:37:41.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.075+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc600005560 con 0x7fc610103ff0 2024-09-12T03:37:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.076+0000 7fc60cc00640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc5f40a2cf0 0x7fc5f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.076+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9525+0+0 (secure 0 0 0) 0x7fc6000cf810 con 0x7fc610103ff0 2024-09-12T03:37:41.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.076+0000 7fc606a00640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc5f40a2cf0 0x7fc5f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:41.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.077+0000 7fc606a00640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc5f40a2cf0 0x7fc5f40a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc61010ab00 tx=0x7fc5f8005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:41.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.080+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6000051d0 con 0x7fc610103ff0 2024-09-12T03:37:41.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.277+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc61019b940 con 0x7fc610103ff0 2024-09-12T03:37:41.278 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:41 smithi050 ceph-mon[30378]: pgmap v326: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:41.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.278+0000 7fc60cc00640 1 -- 172.21.15.50:0/866167941 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v104) ==== 81+0+941 (secure 0 0 0) 0x7fc600084e20 con 0x7fc610103ff0 2024-09-12T03:37:41.278 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3692206583 2024-09-13T03:27:11.145278+0000 2024-09-12T03:37:41.278 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/308265429 2024-09-13T03:27:11.145278+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/63385930 2024-09-13T03:27:11.145278+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1226501005 2024-09-13T03:16:45.524611+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/1939431272 2024-09-13T03:16:45.524611+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2850856182 2024-09-13T03:17:12.653017+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/1939431272 2024-09-13T03:16:45.524611+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1765792264 2024-09-13T03:16:45.524611+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1993742081 2024-09-13T03:27:11.145278+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3574998620 2024-09-13T03:17:12.653017+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/1363145359 2024-09-13T03:17:12.653017+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2127938282 2024-09-13T03:16:45.524611+0000 2024-09-12T03:37:41.279 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3889049657 2024-09-13T03:27:11.145278+0000 2024-09-12T03:37:41.280 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2850856182 2024-09-13T03:17:12.653017+0000 2024-09-12T03:37:41.280 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/63385930 2024-09-13T03:27:11.145278+0000 2024-09-12T03:37:41.280 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3330091767 2024-09-13T03:17:12.653017+0000 2024-09-12T03:37:41.280 INFO:teuthology.orchestra.run.smithi050.stderr:listed 16 entries 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc5f40a2cf0 msgr2=0x7fc5f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc5f40a2cf0 0x7fc5f40a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc61010ab00 tx=0x7fc5f8005f90 comp rx=0 tx=0).stop 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc610103ff0 msgr2=0x7fc61019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc610103ff0 0x7fc61019a5e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e9f0 tx=0x7fc60000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 shutdown_connections 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc610109120 0x7fc61019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fc5f40a2cf0 0x7fc5f40a51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6101049a0 0x7fc61019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 --2- 172.21.15.50:0/866167941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc610103ff0 0x7fc61019a5e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:41.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 >> 172.21.15.50:0/866167941 conn(0x7fc6100ffa90 msgr2=0x7fc6101005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:41.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 shutdown_connections 2024-09-12T03:37:41.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:41.281+0000 7fc616600640 1 -- 172.21.15.50:0/866167941 wait complete. 2024-09-12T03:37:41.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:41 smithi142 ceph-mon[35329]: pgmap v326: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:41.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:41 smithi082 ceph-mon[35046]: pgmap v326: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:42.029 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:37:42.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:37:42.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:37:42.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:37:42.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:37:42.142 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:37:42.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:37:42.234 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-09-12T03:37:42.285 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:37:42.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:37:42.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:37:42.334 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:42 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/866167941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:37:42.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:37:42.344 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-09-12T03:37:42.353 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:42 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/866167941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:37:42.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:42 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/866167941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:37:42.647 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:37:42.657 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:37:42.658 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:37:42.658 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:37:43.036 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:43.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:43 smithi050 ceph-mon[30378]: pgmap v327: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:43.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:43 smithi082 ceph-mon[35046]: pgmap v327: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:43.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:43 smithi142 ceph-mon[35329]: pgmap v327: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 -- 172.21.15.50:0/2656387506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c103ff0 msgr2=0x7faa6c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 --2- 172.21.15.50:0/2656387506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c103ff0 0x7faa6c1043d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009900 tx=0x7faa5c02a410 comp rx=0 tx=0).stop 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 -- 172.21.15.50:0/2656387506 shutdown_connections 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 --2- 172.21.15.50:0/2656387506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa6c109120 0x7faa6c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 --2- 172.21.15.50:0/2656387506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c1049a0 0x7faa6c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 --2- 172.21.15.50:0/2656387506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c103ff0 0x7faa6c1043d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 -- 172.21.15.50:0/2656387506 >> 172.21.15.50:0/2656387506 conn(0x7faa6c0ffa90 msgr2=0x7faa6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 -- 172.21.15.50:0/2656387506 shutdown_connections 2024-09-12T03:37:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.560+0000 7faa73a00640 1 -- 172.21.15.50:0/2656387506 wait complete. 2024-09-12T03:37:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.561+0000 7faa73a00640 1 Processor -- start 2024-09-12T03:37:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.561+0000 7faa73a00640 1 -- start start 2024-09-12T03:37:43.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa73a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c103ff0 0x7faa6c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa73a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 0x7faa6c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa73a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa6c109120 0x7faa6c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa73a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c114170 con 0x7faa6c1049a0 2024-09-12T03:37:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa73a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c113ff0 con 0x7faa6c103ff0 2024-09-12T03:37:43.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa73a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1142f0 con 0x7faa6c109120 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa63e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 0x7faa6c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa71000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c103ff0 0x7faa6c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa63e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 0x7faa6c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34076/0 (socket says 172.21.15.50:34076) 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa63e00640 1 -- 172.21.15.50:0/2269360884 learned_addr learned my addr 172.21.15.50:0/2269360884 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.562+0000 7faa71a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa6c109120 0x7faa6c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa63e00640 1 -- 172.21.15.50:0/2269360884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa6c109120 msgr2=0x7faa6c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa63e00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa6c109120 0x7faa6c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa63e00640 1 -- 172.21.15.50:0/2269360884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c103ff0 msgr2=0x7faa6c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa63e00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c103ff0 0x7faa6c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa63e00640 1 -- 172.21.15.50:0/2269360884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c19f620 con 0x7faa6c1049a0 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa71000640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c103ff0 0x7faa6c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:43.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.563+0000 7faa63e00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 0x7faa6c19abb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa5400d950 tx=0x7faa5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.564+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54017030 con 0x7faa6c1049a0 2024-09-12T03:37:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.564+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54004020 con 0x7faa6c1049a0 2024-09-12T03:37:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.564+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c19f910 con 0x7faa6c1049a0 2024-09-12T03:37:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.564+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540047a0 con 0x7faa6c1049a0 2024-09-12T03:37:43.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.564+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1a71f0 con 0x7faa6c1049a0 2024-09-12T03:37:43.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.565+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c1043d0 con 0x7faa6c1049a0 2024-09-12T03:37:43.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.567+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faa540041d0 con 0x7faa6c1049a0 2024-09-12T03:37:43.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.569+0000 7faa6a000640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7faa500a2d70 0x7faa500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:43.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.569+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9525+0+0 (secure 0 0 0) 0x7faa540cf810 con 0x7faa6c1049a0 2024-09-12T03:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.569+0000 7faa71000640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7faa500a2d70 0x7faa500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:43.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.570+0000 7faa71000640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7faa500a2d70 0x7faa500a5230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000c00 tx=0x7faa5c031510 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:43.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.571+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa5409c060 con 0x7faa6c1049a0 2024-09-12T03:37:43.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.802+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa6c19b9d0 con 0x7faa6c1049a0 2024-09-12T03:37:43.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.802+0000 7faa6a000640 1 -- 172.21.15.50:0/2269360884 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11915 (secure 0 0 0) 0x7faa5409b960 con 0x7faa6c1049a0 2024-09-12T03:37:43.802 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:37:43.802 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":38,"btime":"2024-09-12T03:37:25:698130+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:37:25.024403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25376":{"gid":25376,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1308550298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1308550298},{"type":"v1","addr":"172.21.15.142:6835","nonce":1308550298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24526,24508]},"id":1}]} 2024-09-12T03:37:43.803 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 38 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7faa500a2d70 msgr2=0x7faa500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7faa500a2d70 0x7faa500a5230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000c00 tx=0x7faa5c031510 comp rx=0 tx=0).stop 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 msgr2=0x7faa6c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 0x7faa6c19abb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa5400d950 tx=0x7faa5400de10 comp rx=0 tx=0).stop 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 shutdown_connections 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa6c109120 0x7faa6c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7faa500a2d70 0x7faa500a5230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa6c1049a0 0x7faa6c19abb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 --2- 172.21.15.50:0/2269360884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa6c103ff0 0x7faa6c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 >> 172.21.15.50:0/2269360884 conn(0x7faa6c0ffa90 msgr2=0x7faa6c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 shutdown_connections 2024-09-12T03:37:43.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:43.805+0000 7faa73a00640 1 -- 172.21.15.50:0/2269360884 wait complete. 2024-09-12T03:37:44.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:44 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2269360884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:37:44.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:44 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2269360884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:37:44.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:37:44.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:44 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2269360884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:37:44.921 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:45.340 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:45 smithi050 ceph-mon[30378]: pgmap v328: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:37:45.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:45 smithi082 ceph-mon[35046]: pgmap v328: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:37:45.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 -- 172.21.15.50:0/780271802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c1049a0 msgr2=0x7fad5c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 --2- 172.21.15.50:0/780271802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c1049a0 0x7fad5c1089f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad44009900 tx=0x7fad4402a410 comp rx=0 tx=0).stop 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 -- 172.21.15.50:0/780271802 shutdown_connections 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 --2- 172.21.15.50:0/780271802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c109120 0x7fad5c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 --2- 172.21.15.50:0/780271802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c1049a0 0x7fad5c1089f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 --2- 172.21.15.50:0/780271802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c103ff0 0x7fad5c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 -- 172.21.15.50:0/780271802 >> 172.21.15.50:0/780271802 conn(0x7fad5c0ffa90 msgr2=0x7fad5c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 -- 172.21.15.50:0/780271802 shutdown_connections 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.453+0000 7fad61a00640 1 -- 172.21.15.50:0/780271802 wait complete. 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.454+0000 7fad61a00640 1 Processor -- start 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.454+0000 7fad61a00640 1 -- start start 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad61a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c103ff0 0x7fad5c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:45.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad61a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c1049a0 0x7fad5c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad61a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 0x7fad5c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad61a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c114140 con 0x7fad5c109120 2024-09-12T03:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad61a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c113fc0 con 0x7fad5c103ff0 2024-09-12T03:37:45.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad61a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1142c0 con 0x7fad5c1049a0 2024-09-12T03:37:45.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad5b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 0x7fad5c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:45.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad5aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c103ff0 0x7fad5c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:45.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad5b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 0x7fad5c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34100/0 (socket says 172.21.15.50:34100) 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad5b400640 1 -- 172.21.15.50:0/4130499630 learned_addr learned my addr 172.21.15.50:0/4130499630 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.455+0000 7fad53e00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c1049a0 0x7fad5c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5aa00640 1 -- 172.21.15.50:0/4130499630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c1049a0 msgr2=0x7fad5c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5aa00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c1049a0 0x7fad5c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5aa00640 1 -- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 msgr2=0x7fad5c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5aa00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 0x7fad5c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5aa00640 1 -- 172.21.15.50:0/4130499630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c19f5d0 con 0x7fad5c103ff0 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5b400640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 0x7fad5c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad53e00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c1049a0 0x7fad5c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad5aa00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c103ff0 0x7fad5c19a620 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e9f0 tx=0x7fad4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00ccc0 con 0x7fad5c103ff0 2024-09-12T03:37:45.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c19f8c0 con 0x7fad5c103ff0 2024-09-12T03:37:45.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4c005020 con 0x7fad5c103ff0 2024-09-12T03:37:45.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.456+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0049e0 con 0x7fad5c103ff0 2024-09-12T03:37:45.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.457+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c1a71a0 con 0x7fad5c103ff0 2024-09-12T03:37:45.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.457+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c1043d0 con 0x7fad5c103ff0 2024-09-12T03:37:45.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.460+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fad4c005560 con 0x7fad5c103ff0 2024-09-12T03:37:45.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.461+0000 7fad53400640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fad400a2d70 0x7fad400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:45.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.461+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9525+0+0 (secure 0 0 0) 0x7fad4c0cf860 con 0x7fad5c103ff0 2024-09-12T03:37:45.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.461+0000 7fad53e00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fad400a2d70 0x7fad400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:45.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.462+0000 7fad53e00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fad400a2d70 0x7fad400a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad44009230 tx=0x7fad440058d0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:45.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.463+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad4c0051d0 con 0x7fad5c103ff0 2024-09-12T03:37:45.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:45 smithi142 ceph-mon[35329]: pgmap v328: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:37:45.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.689+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad5c10c3c0 con 0x7fad5c103ff0 2024-09-12T03:37:45.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.690+0000 7fad53400640 1 -- 172.21.15.50:0/4130499630 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11915 (secure 0 0 0) 0x7fad4c09c0c0 con 0x7fad5c103ff0 2024-09-12T03:37:45.691 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:37:45.691 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":38,"btime":"2024-09-12T03:37:25:698130+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3398998523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3398998523},{"type":"v1","addr":"172.21.15.50:6835","nonce":3398998523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3736856191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3736856191},{"type":"v1","addr":"172.21.15.50:6839","nonce":3736856191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/2806488600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":2806488600},{"type":"v1","addr":"172.21.15.50:6841","nonce":2806488600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/196275210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":196275210},{"type":"v1","addr":"172.21.15.82:6833","nonce":196275210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1399602993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1399602993},{"type":"v1","addr":"172.21.15.142:6833","nonce":1399602993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3806772200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3806772200},{"type":"v1","addr":"172.21.15.50:6837","nonce":3806772200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:24:32.635781+0000","modified":"2024-09-12T03:37:25.024403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24526,"mds_2":24508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1268137141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1268137141},{"type":"v1","addr":"172.21.15.82:6835","nonce":1268137141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6837/4279679346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":4279679346},{"type":"v1","addr":"172.21.15.82:6837","nonce":4279679346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/2093449893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2093449893},{"type":"v1","addr":"172.21.15.142:6837","nonce":2093449893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.142:6839/3589126725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3589126725},{"type":"v1","addr":"172.21.15.142:6839","nonce":3589126725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2348373979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2348373979},{"type":"v1","addr":"172.21.15.82:6839","nonce":2348373979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25376":{"gid":25376,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1308550298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1308550298},{"type":"v1","addr":"172.21.15.142:6835","nonce":1308550298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24526,24508]},"id":1}]} 2024-09-12T03:37:45.691 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 38 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fad400a2d70 msgr2=0x7fad400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fad400a2d70 0x7fad400a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad44009230 tx=0x7fad440058d0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c103ff0 msgr2=0x7fad5c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c103ff0 0x7fad5c19a620 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e9f0 tx=0x7fad4c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 shutdown_connections 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fad400a2d70 0x7fad400a5230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad5c109120 0x7fad5c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad5c1049a0 0x7fad5c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 --2- 172.21.15.50:0/4130499630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad5c103ff0 0x7fad5c19a620 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:45.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 >> 172.21.15.50:0/4130499630 conn(0x7fad5c0ffa90 msgr2=0x7fad5c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:45.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 shutdown_connections 2024-09-12T03:37:45.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:45.692+0000 7fad61a00640 1 -- 172.21.15.50:0/4130499630 wait complete. 2024-09-12T03:37:46.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:46 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/4130499630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:37:46.410 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:46 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/4130499630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:37:46.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:37:46.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:46 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/4130499630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:37:46.825 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:47.214 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:47 smithi050 ceph-mon[30378]: pgmap v329: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:37:47.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 -- 172.21.15.50:0/792705366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 msgr2=0x7f9800104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:47.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 --2- 172.21.15.50:0/792705366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f9800104210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009900 tx=0x7f97f402a410 comp rx=0 tx=0).stop 2024-09-12T03:37:47.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 -- 172.21.15.50:0/792705366 shutdown_connections 2024-09-12T03:37:47.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 --2- 172.21.15.50:0/792705366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9800108f60 0x7f9800110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 --2- 172.21.15.50:0/792705366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98001047e0 0x7f9800108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 --2- 172.21.15.50:0/792705366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f9800104210 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 -- 172.21.15.50:0/792705366 >> 172.21.15.50:0/792705366 conn(0x7f98000ff8d0 msgr2=0x7f9800101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:47.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 -- 172.21.15.50:0/792705366 shutdown_connections 2024-09-12T03:37:47.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.327+0000 7f9806600640 1 -- 172.21.15.50:0/792705366 wait complete. 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.328+0000 7f9806600640 1 Processor -- start 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.328+0000 7f9806600640 1 -- start start 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9806600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f980019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9806600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98001047e0 0x7f980019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9806600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9800108f60 0x7f980019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9806600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9800114010 con 0x7f9800103e30 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9806600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9800113e90 con 0x7f98001047e0 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9806600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9800114190 con 0x7f9800108f60 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f980019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f980019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34112/0 (socket says 172.21.15.50:34112) 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 -- 172.21.15.50:0/1988802900 learned_addr learned my addr 172.21.15.50:0/1988802900 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:47.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805c00640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9800108f60 0x7f980019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:47.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f97ffe00640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98001047e0 0x7f980019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:47.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 -- 172.21.15.50:0/1988802900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9800108f60 msgr2=0x7f980019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9800108f60 0x7f980019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 -- 172.21.15.50:0/1988802900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98001047e0 msgr2=0x7f980019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98001047e0 0x7f980019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.329+0000 7f9805200640 1 -- 172.21.15.50:0/1988802900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980019f450 con 0x7f9800103e30 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.330+0000 7f9805200640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f980019a4a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f97f4004770 tx=0x7f97f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.330+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f403f070 con 0x7f9800103e30 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.330+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f4031ca0 con 0x7f9800103e30 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.330+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4035030 con 0x7f9800103e30 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.330+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980019f6e0 con 0x7f9800103e30 2024-09-12T03:37:47.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.330+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980019fb70 con 0x7f9800103e30 2024-09-12T03:37:47.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.332+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9800104210 con 0x7f9800103e30 2024-09-12T03:37:47.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.333+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f97f4042070 con 0x7f9800103e30 2024-09-12T03:37:47.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.335+0000 7f97fd600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f97e80a2d70 0x7f97e80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:47.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.335+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9525+0+0 (secure 0 0 0) 0x7f97f40efb60 con 0x7f9800103e30 2024-09-12T03:37:47.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.335+0000 7f97ffe00640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f97e80a2d70 0x7f97e80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:47.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.336+0000 7f97ffe00640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f97e80a2d70 0x7f97e80a5230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98000ffd60 tx=0x7f97ec00b410 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:47.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.339+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97f40bc330 con 0x7f9800103e30 2024-09-12T03:37:47.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:47 smithi082 ceph-mon[35046]: pgmap v329: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:37:47.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.526+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f980010c1a0 con 0x7f9800103e30 2024-09-12T03:37:47.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.528+0000 7f97fd600640 1 -- 172.21.15.50:0/1988802900 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+25379 (secure 0 0 0) 0x7f97f40bbc30 con 0x7f9800103e30 2024-09-12T03:37:47.529 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-12T03:37:47.529 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":104,"fsid":"389f22da-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:16:17.849714+0000","modified":"2024-09-12T03:37:23.694212+0000","last_up_change":"2024-09-12T03:23:35.215010+0000","last_in_change":"2024-09-12T03:23:10.879781+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:19:33.283337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:24:28.237350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:24:30.608939+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:24:39.200529+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"650c6e13-7a10-4d87-abd0-ed5b66b22521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6803","nonce":894870617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6805","nonce":894870617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6809","nonce":894870617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":894870617},{"type":"v1","addr":"172.21.15.50:6807","nonce":894870617}]},"public_addr":"172.21.15.50:6803/894870617","cluster_addr":"172.21.15.50:6805/894870617","heartbeat_back_addr":"172.21.15.50:6809/894870617","heartbeat_front_addr":"172.21.15.50:6807/894870617","state":["exists","up"]},{"osd":1,"uuid":"989d2e6e-def7-4a97-a83f-da0168e02a90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6801","nonce":987070931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6803","nonce":987070931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6807","nonce":987070931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":987070931},{"type":"v1","addr":"172.21.15.82:6805","nonce":987070931}]},"public_addr":"172.21.15.82:6801/987070931","cluster_addr":"172.21.15.82:6803/987070931","heartbeat_back_addr":"172.21.15.82:6807/987070931","heartbeat_front_addr":"172.21.15.82:6805/987070931","state":["exists","up"]},{"osd":2,"uuid":"b5565f10-86c0-4440-9256-ae2f1729cfb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6801","nonce":1535997813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6803","nonce":1535997813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6807","nonce":1535997813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1535997813},{"type":"v1","addr":"172.21.15.142:6805","nonce":1535997813}]},"public_addr":"172.21.15.142:6801/1535997813","cluster_addr":"172.21.15.142:6803/1535997813","heartbeat_back_addr":"172.21.15.142:6807/1535997813","heartbeat_front_addr":"172.21.15.142:6805/1535997813","state":["exists","up"]},{"osd":3,"uuid":"a331fa56-9a0a-4823-bd51-3d23adb41a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6811","nonce":2709499694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6813","nonce":2709499694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6817","nonce":2709499694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2709499694},{"type":"v1","addr":"172.21.15.50:6815","nonce":2709499694}]},"public_addr":"172.21.15.50:6811/2709499694","cluster_addr":"172.21.15.50:6813/2709499694","heartbeat_back_addr":"172.21.15.50:6817/2709499694","heartbeat_front_addr":"172.21.15.50:6815/2709499694","state":["exists","up"]},{"osd":4,"uuid":"61ff9884-cde7-4179-836b-0acc8b7a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6809","nonce":2469208064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6811","nonce":2469208064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6815","nonce":2469208064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2469208064},{"type":"v1","addr":"172.21.15.82:6813","nonce":2469208064}]},"public_addr":"172.21.15.82:6809/2469208064","cluster_addr":"172.21.15.82:6811/2469208064","heartbeat_back_addr":"172.21.15.82:6815/2469208064","heartbeat_front_addr":"172.21.15.82:6813/2469208064","state":["exists","up"]},{"osd":5,"uuid":"0636aa4c-98aa-4b31-86ea-d77fc89be048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6809","nonce":361252032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6811","nonce":361252032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6815","nonce":361252032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":361252032},{"type":"v1","addr":"172.21.15.142:6813","nonce":361252032}]},"public_addr":"172.21.15.142:6809/361252032","cluster_addr":"172.21.15.142:6811/361252032","heartbeat_back_addr":"172.21.15.142:6815/361252032","heartbeat_front_addr":"172.21.15.142:6813/361252032","state":["exists","up"]},{"osd":6,"uuid":"cd0f4cc5-c370-4f89-8332-b28096839552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6819","nonce":3669387354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6821","nonce":3669387354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6825","nonce":3669387354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3669387354},{"type":"v1","addr":"172.21.15.50:6823","nonce":3669387354}]},"public_addr":"172.21.15.50:6819/3669387354","cluster_addr":"172.21.15.50:6821/3669387354","heartbeat_back_addr":"172.21.15.50:6825/3669387354","heartbeat_front_addr":"172.21.15.50:6823/3669387354","state":["exists","up"]},{"osd":7,"uuid":"a73508ea-e10f-466a-8ce6-fcdffbb2f047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6817","nonce":704851047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6819","nonce":704851047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6823","nonce":704851047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":704851047},{"type":"v1","addr":"172.21.15.82:6821","nonce":704851047}]},"public_addr":"172.21.15.82:6817/704851047","cluster_addr":"172.21.15.82:6819/704851047","heartbeat_back_addr":"172.21.15.82:6823/704851047","heartbeat_front_addr":"172.21.15.82:6821/704851047","state":["exists","up"]},{"osd":8,"uuid":"1fe07bf7-a040-4ddc-b175-de41dae683c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6817","nonce":1190406046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6819","nonce":1190406046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6823","nonce":1190406046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1190406046},{"type":"v1","addr":"172.21.15.142:6821","nonce":1190406046}]},"public_addr":"172.21.15.142:6817/1190406046","cluster_addr":"172.21.15.142:6819/1190406046","heartbeat_back_addr":"172.21.15.142:6823/1190406046","heartbeat_front_addr":"172.21.15.142:6821/1190406046","state":["exists","up"]},{"osd":9,"uuid":"be1e8d37-b1a4-4f1f-a019-efa72dcaacaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6827","nonce":1350383902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6829","nonce":1350383902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6833","nonce":1350383902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1350383902},{"type":"v1","addr":"172.21.15.50:6831","nonce":1350383902}]},"public_addr":"172.21.15.50:6827/1350383902","cluster_addr":"172.21.15.50:6829/1350383902","heartbeat_back_addr":"172.21.15.50:6833/1350383902","heartbeat_front_addr":"172.21.15.50:6831/1350383902","state":["exists","up"]},{"osd":10,"uuid":"2660198c-5848-4738-910b-711c291758a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6825","nonce":1817816537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6827","nonce":1817816537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6831","nonce":1817816537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1817816537},{"type":"v1","addr":"172.21.15.82:6829","nonce":1817816537}]},"public_addr":"172.21.15.82:6825/1817816537","cluster_addr":"172.21.15.82:6827/1817816537","heartbeat_back_addr":"172.21.15.82:6831/1817816537","heartbeat_front_addr":"172.21.15.82:6829/1817816537","state":["exists","up"]},{"osd":11,"uuid":"7d5befe3-b567-4cd2-b31a-a8d06d402568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6825","nonce":572198399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6827","nonce":572198399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6831","nonce":572198399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":572198399},{"type":"v1","addr":"172.21.15.142:6829","nonce":572198399}]},"public_addr":"172.21.15.142:6825/572198399","cluster_addr":"172.21.15.142:6827/572198399","heartbeat_back_addr":"172.21.15.142:6831/572198399","heartbeat_front_addr":"172.21.15.142:6829/572198399","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:18:40.737633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:07.078933+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:19:31.560257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:01.950979+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:28.333997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:20:52.137773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:20.263493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:47.073340+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:11.981945+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:41.863930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:07.538989+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:33.184612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:0/3692206583":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/308265429":"2024-09-13T03:27:11.145278+0000","172.21.15.50:6801/63385930":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/1226501005":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6800/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:6801/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6801/1939431272":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1765792264":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/1993742081":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/3574998620":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/1363145359":"2024-09-13T03:17:12.653017+0000","172.21.15.50:0/2127938282":"2024-09-13T03:16:45.524611+0000","172.21.15.50:0/3889049657":"2024-09-13T03:27:11.145278+0000","172.21.15.50:6800/2850856182":"2024-09-13T03:17:12.653017+0000","172.21.15.50:6800/63385930":"2024-09-13T03:27:11.145278+0000","172.21.15.50:0/3330091767":"2024-09-13T03:17:12.653017+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f97e80a2d70 msgr2=0x7f97e80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f97e80a2d70 0x7f97e80a5230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98000ffd60 tx=0x7f97ec00b410 comp rx=0 tx=0).stop 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 msgr2=0x7f980019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f980019a4a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f97f4004770 tx=0x7f97f4003c10 comp rx=0 tx=0).stop 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 shutdown_connections 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9800108f60 0x7f980019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98001047e0 0x7f980019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f97e80a2d70 0x7f97e80a5230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 --2- 172.21.15.50:0/1988802900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9800103e30 0x7f980019a4a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 >> 172.21.15.50:0/1988802900 conn(0x7f98000ff8d0 msgr2=0x7f9800100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 shutdown_connections 2024-09-12T03:37:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:47.530+0000 7f9806600640 1 -- 172.21.15.50:0/1988802900 wait complete. 2024-09-12T03:37:47.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:47 smithi142 ceph-mon[35329]: pgmap v329: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:37:48.164 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:48 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/1988802900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:37:48.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:37:48.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:48 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/1988802900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:37:48.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:48 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/1988802900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:37:48.618 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:49.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 -- 172.21.15.50:0/1858750526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0901027a0 msgr2=0x7ff0901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:49.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 --2- 172.21.15.50:0/1858750526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0901027a0 0x7ff0901067f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff078009860 tx=0x7ff07802a340 comp rx=0 tx=0).stop 2024-09-12T03:37:49.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 -- 172.21.15.50:0/1858750526 shutdown_connections 2024-09-12T03:37:49.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 --2- 172.21.15.50:0/1858750526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff090106f20 0x7ff09010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 --2- 172.21.15.50:0/1858750526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0901027a0 0x7ff0901067f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 --2- 172.21.15.50:0/1858750526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 -- 172.21.15.50:0/1858750526 >> 172.21.15.50:0/1858750526 conn(0x7ff0900fd8b0 msgr2=0x7ff0900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:49.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 -- 172.21.15.50:0/1858750526 shutdown_connections 2024-09-12T03:37:49.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.179+0000 7ff096c00640 1 -- 172.21.15.50:0/1858750526 wait complete. 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.180+0000 7ff096c00640 1 Processor -- start 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.180+0000 7ff096c00640 1 -- start start 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff096c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff096c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0901027a0 0x7ff090198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff096c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff090106f20 0x7ff09019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff096c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff090111f80 con 0x7ff090101df0 2024-09-12T03:37:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff096c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff090111e00 con 0x7ff0901027a0 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff096c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff090112100 con 0x7ff090106f20 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49384/0 (socket says 172.21.15.50:49384) 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 -- 172.21.15.50:0/961565975 learned_addr learned my addr 172.21.15.50:0/961565975 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff087e00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0901027a0 0x7ff090198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff094c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff090106f20 0x7ff09019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 -- 172.21.15.50:0/961565975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff090106f20 msgr2=0x7ff09019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff090106f20 0x7ff09019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 -- 172.21.15.50:0/961565975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0901027a0 msgr2=0x7ff090198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0901027a0 0x7ff090198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.181+0000 7ff08fe00640 1 -- 172.21.15.50:0/961565975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09019d390 con 0x7ff090101df0 2024-09-12T03:37:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff087e00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0901027a0 0x7ff090198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff08fe00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901983e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff08000e9c0 tx=0x7ff08000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08000ccc0 con 0x7ff090101df0 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff080005500 con 0x7ff090101df0 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09019d680 con 0x7ff090101df0 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080010430 con 0x7ff090101df0 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.182+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a4f60 con 0x7ff090101df0 2024-09-12T03:37:49.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.184+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0901021d0 con 0x7ff090101df0 2024-09-12T03:37:49.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.185+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff0800046c0 con 0x7ff090101df0 2024-09-12T03:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.187+0000 7ff08d600640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7ff0740a2cf0 0x7ff0740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.187+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9525+0+0 (secure 0 0 0) 0x7ff0800cfb70 con 0x7ff090101df0 2024-09-12T03:37:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.187+0000 7ff087e00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7ff0740a2cf0 0x7ff0740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:49.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.188+0000 7ff087e00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7ff0740a2cf0 0x7ff0740a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a930 tx=0x7ff0780056d0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:49.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.191+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff08009c340 con 0x7ff090101df0 2024-09-12T03:37:49.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-mon[35046]: pgmap v330: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T03:37:49.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:49 smithi050 ceph-mon[30378]: pgmap v330: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T03:37:49.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.417+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff090109b70 con 0x7ff090101df0 2024-09-12T03:37:49.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.420+0000 7ff08d600640 1 -- 172.21.15.50:0/961565975 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7ff08009bc40 con 0x7ff090101df0 2024-09-12T03:37:49.421 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:37:49.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.422+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7ff0740a2cf0 msgr2=0x7ff0740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.422+0000 7ff096c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7ff0740a2cf0 0x7ff0740a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a930 tx=0x7ff0780056d0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.422+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 msgr2=0x7ff0901983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.422+0000 7ff096c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901983e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff08000e9c0 tx=0x7ff08000ee80 comp rx=0 tx=0).stop 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 shutdown_connections 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff090106f20 0x7ff09019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0901027a0 0x7ff090198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7ff0740a2cf0 0x7ff0740a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 --2- 172.21.15.50:0/961565975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff090101df0 0x7ff0901983e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 >> 172.21.15.50:0/961565975 conn(0x7ff0900fd8b0 msgr2=0x7ff0900feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:49.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 shutdown_connections 2024-09-12T03:37:49.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:49.423+0000 7ff096c00640 1 -- 172.21.15.50:0/961565975 wait complete. 2024-09-12T03:37:49.469 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-mon[35329]: pgmap v330: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T03:37:49.762 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:49 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a[30374]: 2024-09-12T03:37:49.418+0000 7f5299400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i[66137]: ignoring --setuser ceph since I am not root 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i[66137]: ignoring --setgroup ceph since I am not root 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-mds[66141]: -- 172.21.15.142:0/3609107856 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ebabf83860 con 0x55ebabf61000 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i[66137]: starting mds.i at 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c[65834]: ignoring --setuser ceph since I am not root 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c[65834]: ignoring --setgroup ceph since I am not root 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-mds[65838]: -- 172.21.15.142:0/403884671 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc330a9860 con 0x55dc33087400 2024-09-12T03:37:49.794 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c[65834]: starting mds.c at 2024-09-12T03:37:49.795 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[69849]: ignoring --setuser ceph since I am not root 2024-09-12T03:37:49.795 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[69849]: ignoring --setgroup ceph since I am not root 2024-09-12T03:37:49.795 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-mds[69853]: -- 172.21.15.142:0/3551536725 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628d34df860 con 0x5628d34bc800 2024-09-12T03:37:49.795 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:37:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[69849]: starting mds.f at 2024-09-12T03:37:49.854 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e[64264]: ignoring --setuser ceph since I am not root 2024-09-12T03:37:49.854 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e[64264]: ignoring --setgroup ceph since I am not root 2024-09-12T03:37:49.854 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-mds[64268]: -- 172.21.15.82:0/908469825 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b20cf59860 con 0x55b20cf36800 2024-09-12T03:37:49.854 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e[64264]: starting mds.e at 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h[64574]: ignoring --setuser ceph since I am not root 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h[64574]: ignoring --setgroup ceph since I am not root 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-mds[64578]: -- 172.21.15.82:0/1841308515 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625460a3860 con 0x562546081400 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h[64574]: starting mds.h at 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: ignoring --setuser ceph since I am not root 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: ignoring --setgroup ceph since I am not root 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-mds[64881]: -- 172.21.15.82:0/4291428755 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561c5b618680 con 0x561c5b5ed400 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-mds[64881]: -- 172.21.15.82:0/4291428755 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c5b60f860 con 0x561c5b5ed400 2024-09-12T03:37:49.855 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:37:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: starting mds.b at 2024-09-12T03:37:50.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: pgmap v331: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/961565975' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/961565975' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:37:50.354 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:50 smithi082 ceph-mon[35046]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:37:50.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: pgmap v331: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:37:50.414 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/961565975' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:37:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:37:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:37:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/961565975' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:37:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:37:50.415 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:50 smithi050 ceph-mon[30378]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:37:50.502 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:50.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: pgmap v331: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:37:50.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/961565975' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:37:50.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:37:50.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:37:50.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/961565975' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:37:50.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:37:50.544 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:50 smithi142 ceph-mon[35329]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 -- 172.21.15.50:0/2092130090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc109140 msgr2=0x7fd3dc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/2092130090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc109140 0x7fd3dc110cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009ca0 tx=0x7fd3d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 -- 172.21.15.50:0/2092130090 shutdown_connections 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/2092130090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc109140 0x7fd3dc110cc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/2092130090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/2092130090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3dc104010 0x7fd3dc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 -- 172.21.15.50:0/2092130090 >> 172.21.15.50:0/2092130090 conn(0x7fd3dc0ffa50 msgr2=0x7fd3dc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 -- 172.21.15.50:0/2092130090 shutdown_connections 2024-09-12T03:37:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.089+0000 7fd3e1a00640 1 -- 172.21.15.50:0/2092130090 wait complete. 2024-09-12T03:37:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.090+0000 7fd3e1a00640 1 Processor -- start 2024-09-12T03:37:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.090+0000 7fd3e1a00640 1 -- start start 2024-09-12T03:37:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc104010 0x7fd3dc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3e1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3dc109140 0x7fd3dc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1141c0 con 0x7fd3dc1049c0 2024-09-12T03:37:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc114040 con 0x7fd3dc104010 2024-09-12T03:37:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3e1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc114340 con 0x7fd3dc109140 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3daa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc104010 0x7fd3dc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49408/0 (socket says 172.21.15.50:49408) 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3d3e00640 1 -- 172.21.15.50:0/3138671689 learned_addr learned my addr 172.21.15.50:0/3138671689 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.091+0000 7fd3db400640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3dc109140 0x7fd3dc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3e00640 1 -- 172.21.15.50:0/3138671689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3dc109140 msgr2=0x7fd3dc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3e00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3dc109140 0x7fd3dc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3e00640 1 -- 172.21.15.50:0/3138671689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc104010 msgr2=0x7fd3dc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3e00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc104010 0x7fd3dc19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3e00640 1 -- 172.21.15.50:0/3138671689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19f670 con 0x7fd3dc1049c0 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3daa00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc104010 0x7fd3dc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3e00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc19ac00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400e9f0 tx=0x7fd3c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c400cd80 con 0x7fd3dc1049c0 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c4010910 con 0x7fd3dc1049c0 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19f960 con 0x7fd3dc1049c0 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4018990 con 0x7fd3dc1049c0 2024-09-12T03:37:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.092+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a7240 con 0x7fd3dc1049c0 2024-09-12T03:37:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.093+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc1043f0 con 0x7fd3dc1049c0 2024-09-12T03:37:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.095+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd3c4010ac0 con 0x7fd3dc1049c0 2024-09-12T03:37:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.097+0000 7fd3d3400640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd3c00a2d70 0x7fd3c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.097+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9695+0+0 (secure 0 0 0) 0x7fd3c40cfb10 con 0x7fd3dc1049c0 2024-09-12T03:37:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.097+0000 7fd3daa00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd3c00a2d70 0x7fd3c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.098+0000 7fd3daa00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd3c00a2d70 0x7fd3c00a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009fc0 tx=0x7fd3cc005f90 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.101+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3c409bbf0 con 0x7fd3dc1049c0 2024-09-12T03:37:51.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.323+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd3dc10bdf0 con 0x7fd3dc1049c0 2024-09-12T03:37:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.428+0000 7fd3d3400640 1 -- 172.21.15.50:0/3138671689 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7fd3c409fef0 con 0x7fd3dc1049c0 2024-09-12T03:37:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.430+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd3c00a2d70 msgr2=0x7fd3c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.430+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd3c00a2d70 0x7fd3c00a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009fc0 tx=0x7fd3cc005f90 comp rx=0 tx=0).stop 2024-09-12T03:37:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.430+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 msgr2=0x7fd3dc19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.430+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc19ac00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400e9f0 tx=0x7fd3c400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.430+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 shutdown_connections 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.430+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3dc109140 0x7fd3dc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.431+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fd3c00a2d70 0x7fd3c00a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.431+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3dc1049c0 0x7fd3dc19ac00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.431+0000 7fd3e1a00640 1 --2- 172.21.15.50:0/3138671689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3dc104010 0x7fd3dc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.431+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 >> 172.21.15.50:0/3138671689 conn(0x7fd3dc0ffa50 msgr2=0x7fd3dc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.431+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 shutdown_connections 2024-09-12T03:37:51.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:51.431+0000 7fd3e1a00640 1 -- 172.21.15.50:0/3138671689 wait complete. 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6835/2735315722,v1:172.21.15.82:6838/2735315722] up:boot 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6834/4255083921,v1:172.21.15.82:6837/4255083921] up:boot 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.82:6836/115884093,v1:172.21.15.82:6839/115884093] up:boot 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6834/770295590,v1:172.21.15.142:6835/770295590] up:boot 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6836/2892841511,v1:172.21.15.142:6837/2892841511] up:boot 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: mds.? [v2:172.21.15.142:6838/868736106,v1:172.21.15.142:6839/868736106] up:boot 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:37:51.773 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:37:51.774 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:37:51.774 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:37:51.774 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:51 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3138671689' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:37:51.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6835/2735315722,v1:172.21.15.82:6838/2735315722] up:boot 2024-09-12T03:37:51.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6834/4255083921,v1:172.21.15.82:6837/4255083921] up:boot 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.82:6836/115884093,v1:172.21.15.82:6839/115884093] up:boot 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6834/770295590,v1:172.21.15.142:6835/770295590] up:boot 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6836/2892841511,v1:172.21.15.142:6837/2892841511] up:boot 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: mds.? [v2:172.21.15.142:6838/868736106,v1:172.21.15.142:6839/868736106] up:boot 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:37:51.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:51 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3138671689' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:37:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6835/2735315722,v1:172.21.15.82:6838/2735315722] up:boot 2024-09-12T03:37:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6834/4255083921,v1:172.21.15.82:6837/4255083921] up:boot 2024-09-12T03:37:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.82:6836/115884093,v1:172.21.15.82:6839/115884093] up:boot 2024-09-12T03:37:51.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6834/770295590,v1:172.21.15.142:6835/770295590] up:boot 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6836/2892841511,v1:172.21.15.142:6837/2892841511] up:boot 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: mds.? [v2:172.21.15.142:6838/868736106,v1:172.21.15.142:6839/868736106] up:boot 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='mgr.25157 172.21.15.50:0/3175371238' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:37:51.855 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:51 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3138671689' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:37:52.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:37:52.582 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:52.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:52 smithi050 ceph-mon[30378]: pgmap v333: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-12T03:37:52.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:52 smithi050 ceph-mon[30378]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:37:52.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:52 smithi050 ceph-mon[30378]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:37:52.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:52 smithi050 ceph-mon[30378]: Cluster is now healthy 2024-09-12T03:37:52.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:52 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3138671689' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:37:52.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:52 smithi050 ceph-mon[30378]: fsmap 12 up:standby 2024-09-12T03:37:52.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:52 smithi142 ceph-mon[35329]: pgmap v333: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-12T03:37:52.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:52 smithi142 ceph-mon[35329]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:37:52.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:52 smithi142 ceph-mon[35329]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:37:52.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:52 smithi142 ceph-mon[35329]: Cluster is now healthy 2024-09-12T03:37:52.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:52 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3138671689' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:37:52.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:52 smithi142 ceph-mon[35329]: fsmap 12 up:standby 2024-09-12T03:37:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:52 smithi082 ceph-mon[35046]: pgmap v333: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-12T03:37:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:52 smithi082 ceph-mon[35046]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:37:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:52 smithi082 ceph-mon[35046]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:37:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:52 smithi082 ceph-mon[35046]: Cluster is now healthy 2024-09-12T03:37:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:52 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3138671689' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:37:52.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:52 smithi082 ceph-mon[35046]: fsmap 12 up:standby 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.144+0000 7f216d000640 1 -- 172.21.15.50:0/2115702167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 msgr2=0x7f2168112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.144+0000 7f216d000640 1 --2- 172.21.15.50:0/2115702167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f2168112510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2154009900 tx=0x7f215402a410 comp rx=0 tx=0).stop 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 -- 172.21.15.50:0/2115702167 shutdown_connections 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 --2- 172.21.15.50:0/2115702167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168112a50 0x7f2168114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 --2- 172.21.15.50:0/2115702167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f2168112510 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 --2- 172.21.15.50:0/2115702167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2168076720 0x7f2168076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 -- 172.21.15.50:0/2115702167 >> 172.21.15.50:0/2115702167 conn(0x7f21680ffa50 msgr2=0x7f2168101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 -- 172.21.15.50:0/2115702167 shutdown_connections 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.145+0000 7f216d000640 1 -- 172.21.15.50:0/2115702167 wait complete. 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 Processor -- start 2024-09-12T03:37:53.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 -- start start 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 0x7f21681a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f21681a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2168112a50 0x7f216819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2168118620 con 0x7f2168077040 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f21681184a0 con 0x7f2168112a50 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.146+0000 7f216d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21681187a0 con 0x7f2168076720 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f21681a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 0x7f21681a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 0x7f21681a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.50:39994/0 (socket says 172.21.15.50:39994) 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 -- 172.21.15.50:0/2691025614 learned_addr learned my addr 172.21.15.50:0/2691025614 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2166800640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2168112a50 0x7f216819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:53.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 -- 172.21.15.50:0/2691025614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2168112a50 msgr2=0x7f216819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2168112a50 0x7f216819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 -- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 msgr2=0x7f21681a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f21681a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 -- 172.21.15.50:0/2691025614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216819f020 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165400640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f21681a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.147+0000 7f2165e00640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 0x7f21681a4730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f215800d950 tx=0x7f215800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.148+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158017030 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.148+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216819f310 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.148+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2158004020 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.148+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21580047a0 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.148+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681ab600 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.149+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2168076b00 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.151+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f21580041d0 con 0x7f2168076720 2024-09-12T03:37:53.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.152+0000 7f215ea00640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f214c0a2cf0 0x7f214c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:53.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.152+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9695+0+0 (secure 0 0 0) 0x7f21580cf910 con 0x7f2168076720 2024-09-12T03:37:53.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.152+0000 7f2165400640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f214c0a2cf0 0x7f214c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:53.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.153+0000 7f2165400640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f214c0a2cf0 0x7f214c0a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154031510 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:53.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.154+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f215809c040 con 0x7f2168076720 2024-09-12T03:37:53.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.340+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f216819fe20 con 0x7f2168076720 2024-09-12T03:37:53.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.440+0000 7f215ea00640 1 -- 172.21.15.50:0/2691025614 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7f215809b940 con 0x7f2168076720 2024-09-12T03:37:53.441 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:37:53.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f214c0a2cf0 msgr2=0x7f214c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f214c0a2cf0 0x7f214c0a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2154000c00 tx=0x7f2154031510 comp rx=0 tx=0).stop 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 msgr2=0x7f21681a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 0x7f21681a4730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f215800d950 tx=0x7f215800de10 comp rx=0 tx=0).stop 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 shutdown_connections 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2168112a50 0x7f216819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f214c0a2cf0 0x7f214c0a51b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2168077040 0x7f21681a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 --2- 172.21.15.50:0/2691025614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168076720 0x7f21681a4730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:53.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 >> 172.21.15.50:0/2691025614 conn(0x7f21680ffa50 msgr2=0x7f2168101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 shutdown_connections 2024-09-12T03:37:53.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:53.443+0000 7f216d000640 1 -- 172.21.15.50:0/2691025614 wait complete. 2024-09-12T03:37:53.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:53 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2691025614' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:53.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:53 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:53.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:53 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2691025614' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:53.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:53 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:53.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:53 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2691025614' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:53.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:53 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:54.161 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:37:54.545 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:54.707 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:54 smithi050 ceph-mon[30378]: pgmap v334: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 102 B/s wr, 4 op/s 2024-09-12T03:37:54.708 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:54 smithi050 ceph-mon[30378]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:54.708 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:54 smithi050 ceph-mon[30378]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:37:54.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:54 smithi142 ceph-mon[35329]: pgmap v334: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 102 B/s wr, 4 op/s 2024-09-12T03:37:54.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:54 smithi142 ceph-mon[35329]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:54.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:54 smithi142 ceph-mon[35329]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:37:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:54 smithi082 ceph-mon[35046]: pgmap v334: 129 pgs: 129 active+clean; 134 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 102 B/s wr, 4 op/s 2024-09-12T03:37:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:54 smithi082 ceph-mon[35046]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:54.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:54 smithi082 ceph-mon[35046]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:37:55.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 -- 172.21.15.50:0/1096699691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4109120 msgr2=0x7fb5a4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:55.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 --2- 172.21.15.50:0/1096699691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4109120 0x7fb5a4110ca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009ca0 tx=0x7fb59c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 -- 172.21.15.50:0/1096699691 shutdown_connections 2024-09-12T03:37:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 --2- 172.21.15.50:0/1096699691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4109120 0x7fb5a4110ca0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 --2- 172.21.15.50:0/1096699691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a41049a0 0x7fb5a41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 --2- 172.21.15.50:0/1096699691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a4103ff0 0x7fb5a41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 -- 172.21.15.50:0/1096699691 >> 172.21.15.50:0/1096699691 conn(0x7fb5a40ffa90 msgr2=0x7fb5a4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 -- 172.21.15.50:0/1096699691 shutdown_connections 2024-09-12T03:37:55.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.085+0000 7fb5ab600640 1 -- 172.21.15.50:0/1096699691 wait complete. 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.086+0000 7fb5ab600640 1 Processor -- start 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.086+0000 7fb5ab600640 1 -- start start 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5ab600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 0x7fb5a419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5ab600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a41049a0 0x7fb5a419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5ab600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a4109120 0x7fb5a419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5ab600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4114150 con 0x7fb5a4103ff0 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5ab600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4113fd0 con 0x7fb5a41049a0 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5ab600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41142d0 con 0x7fb5a4109120 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 0x7fb5a419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:55.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 0x7fb5a419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49440/0 (socket says 172.21.15.50:49440) 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 -- 172.21.15.50:0/3368973396 learned_addr learned my addr 172.21.15.50:0/3368973396 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a9600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a4109120 0x7fb5a419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a3e00640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a41049a0 0x7fb5a419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 -- 172.21.15.50:0/3368973396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a4109120 msgr2=0x7fb5a419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a4109120 0x7fb5a419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 -- 172.21.15.50:0/3368973396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a41049a0 msgr2=0x7fb5a419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a41049a0 0x7fb5a419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.087+0000 7fb5a8c00640 1 -- 172.21.15.50:0/3368973396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a419f5e0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.088+0000 7fb5a8c00640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 0x7fb5a419a630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb59800e9f0 tx=0x7fb59800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.088+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59800cd80 con 0x7fb5a4103ff0 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.088+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb598005500 con 0x7fb5a4103ff0 2024-09-12T03:37:55.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.088+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598010690 con 0x7fb5a4103ff0 2024-09-12T03:37:55.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.088+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a419f8d0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.088+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a71b0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.089+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a41043d0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.091+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb5980046c0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.093+0000 7fb5a1600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb58c0a2d70 0x7fb58c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:55.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.093+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9141+0+0 (secure 0 0 0) 0x7fb5980cfaa0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.093+0000 7fb5a3e00640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb58c0a2d70 0x7fb58c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:55.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.094+0000 7fb5a3e00640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb58c0a2d70 0x7fb58c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5a410a9e0 tx=0x7fb59000b410 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:55.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.097+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb59809c3f0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.282+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb5a419b990 con 0x7fb5a4103ff0 2024-09-12T03:37:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.442+0000 7fb5a1600640 1 -- 172.21.15.50:0/3368973396 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7fb59809bcf0 con 0x7fb5a4103ff0 2024-09-12T03:37:55.443 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-09-12T03:37:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.446+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb58c0a2d70 msgr2=0x7fb58c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.446+0000 7fb5ab600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb58c0a2d70 0x7fb58c0a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5a410a9e0 tx=0x7fb59000b410 comp rx=0 tx=0).stop 2024-09-12T03:37:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.446+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 msgr2=0x7fb5a419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:55.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.446+0000 7fb5ab600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 0x7fb5a419a630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb59800e9f0 tx=0x7fb59800edf0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 shutdown_connections 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a4109120 0x7fb5a419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a41049a0 0x7fb5a419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7fb58c0a2d70 0x7fb58c0a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 --2- 172.21.15.50:0/3368973396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5a4103ff0 0x7fb5a419a630 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 >> 172.21.15.50:0/3368973396 conn(0x7fb5a40ffa90 msgr2=0x7fb5a41005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 shutdown_connections 2024-09-12T03:37:55.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:55.447+0000 7fb5ab600640 1 -- 172.21.15.50:0/3368973396 wait complete. 2024-09-12T03:37:55.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:55 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3368973396' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:55.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:55 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3368973396' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:55.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:55 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3368973396' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:56.169 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 389f22da-70b5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-12T03:37:56.565 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:37:56.697 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:56 smithi050 ceph-mon[30378]: pgmap v336: 97 pgs: 97 active+clean; 89 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:37:56.697 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:56 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/3368973396' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:56.697 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:56 smithi050 ceph-mon[30378]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:37:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:56 smithi142 ceph-mon[35329]: pgmap v336: 97 pgs: 97 active+clean; 89 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:37:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:56 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/3368973396' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:56.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:56 smithi142 ceph-mon[35329]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:37:56.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:56 smithi082 ceph-mon[35046]: pgmap v336: 97 pgs: 97 active+clean; 89 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:37:56.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:56 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/3368973396' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:56.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:56 smithi082 ceph-mon[35046]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:37:57.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 -- 172.21.15.50:0/2137595102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c90103ff0 msgr2=0x7f6c901043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:57.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 --2- 172.21.15.50:0/2137595102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c90103ff0 0x7f6c901043d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009900 tx=0x7f6c8002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 -- 172.21.15.50:0/2137595102 shutdown_connections 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 --2- 172.21.15.50:0/2137595102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 0x7f6c90110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 --2- 172.21.15.50:0/2137595102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c901049a0 0x7f6c901089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 --2- 172.21.15.50:0/2137595102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c90103ff0 0x7f6c901043d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 -- 172.21.15.50:0/2137595102 >> 172.21.15.50:0/2137595102 conn(0x7f6c900ffa90 msgr2=0x7f6c90101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 -- 172.21.15.50:0/2137595102 shutdown_connections 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.078+0000 7f6c96600640 1 -- 172.21.15.50:0/2137595102 wait complete. 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 Processor -- start 2024-09-12T03:37:57.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 -- start start 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c90103ff0 0x7f6c9019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 0x7f6c9019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 0x7f6c9019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90114120 con 0x7f6c901049a0 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90113fa0 con 0x7f6c90109120 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.079+0000 7f6c96600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901142a0 con 0x7f6c90103ff0 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 0x7f6c9019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:57.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c90103ff0 0x7f6c9019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 0x7f6c9019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49446/0 (socket says 172.21.15.50:49446) 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 -- 172.21.15.50:0/2557138502 learned_addr learned my addr 172.21.15.50:0/2557138502 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8fe00640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 0x7f6c9019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 -- 172.21.15.50:0/2557138502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c90103ff0 msgr2=0x7f6c9019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c90103ff0 0x7f6c9019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 -- 172.21.15.50:0/2557138502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 msgr2=0x7f6c9019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 0x7f6c9019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 -- 172.21.15.50:0/2557138502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9019f5d0 con 0x7f6c901049a0 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8fe00640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 0x7f6c9019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.080+0000 7f6c8ea00640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 0x7f6c9019ab60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00d950 tx=0x7f6c7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.081+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c017070 con 0x7f6c901049a0 2024-09-12T03:37:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.081+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c009c30 con 0x7f6c901049a0 2024-09-12T03:37:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.081+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9019f8c0 con 0x7f6c901049a0 2024-09-12T03:37:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.081+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00f6b0 con 0x7f6c901049a0 2024-09-12T03:37:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.081+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c901a71a0 con 0x7f6c901049a0 2024-09-12T03:37:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.082+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c901043d0 con 0x7f6c901049a0 2024-09-12T03:37:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.085+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6c7c0109d0 con 0x7f6c901049a0 2024-09-12T03:37:57.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.086+0000 7f6c87e00640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6c740a2cf0 0x7f6c740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:57.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.086+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8582+0+0 (secure 0 0 0) 0x7f6c7c0cea70 con 0x7f6c901049a0 2024-09-12T03:37:57.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.086+0000 7f6c8f400640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6c740a2cf0 0x7f6c740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:57.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.087+0000 7f6c8f400640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6c740a2cf0 0x7f6c740a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009230 tx=0x7f6c800058d0 comp rx=0 tx=0).ready entity=mgr.25157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:57.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.090+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c7c009de0 con 0x7f6c901049a0 2024-09-12T03:37:57.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.277+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c9010bd70 con 0x7f6c901049a0 2024-09-12T03:37:57.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.449+0000 7f6c87e00640 1 -- 172.21.15.50:0/2557138502 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v108) ==== 178+0+0 (secure 0 0 0) 0x7f6c7c09b5f0 con 0x7f6c901049a0 2024-09-12T03:37:57.451 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' removed 2024-09-12T03:37:57.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6c740a2cf0 msgr2=0x7f6c740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6c740a2cf0 0x7f6c740a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009230 tx=0x7f6c800058d0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 msgr2=0x7f6c9019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 0x7f6c9019ab60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00d950 tx=0x7f6c7c00de10 comp rx=0 tx=0).stop 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 shutdown_connections 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c90109120 0x7f6c9019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:6800/682249052,v1:172.21.15.50:6801/682249052] conn(0x7f6c740a2cf0 0x7f6c740a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c901049a0 0x7f6c9019ab60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 --2- 172.21.15.50:0/2557138502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c90103ff0 0x7f6c9019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 >> 172.21.15.50:0/2557138502 conn(0x7f6c900ffa90 msgr2=0x7f6c90100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 shutdown_connections 2024-09-12T03:37:57.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-09-12T03:37:57.454+0000 7f6c96600640 1 -- 172.21.15.50:0/2557138502 wait complete. 2024-09-12T03:37:57.664 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:57 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2557138502' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:57.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:57 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2557138502' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:57.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:57 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2557138502' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:37:58.288 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:37:58.346 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:37:58.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:37:58.406 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:37:58.406 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:37:58.453 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:37:58.469 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:37:58.485 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:37:58.485 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:37:58.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:37:58.547 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:58 smithi050 ceph-mon[30378]: pgmap v338: 33 pgs: 33 active+clean; 89 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:37:58.547 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:58 smithi050 ceph-mon[30378]: from='client.? 172.21.15.50:0/2557138502' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:58.547 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:58 smithi050 ceph-mon[30378]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:37:58.793 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:58 smithi142 ceph-mon[35329]: pgmap v338: 33 pgs: 33 active+clean; 89 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:37:58.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:58 smithi142 ceph-mon[35329]: from='client.? 172.21.15.50:0/2557138502' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:58.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:37:58 smithi142 ceph-mon[35329]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:37:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:58 smithi082 ceph-mon[35046]: pgmap v338: 33 pgs: 33 active+clean; 89 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:37:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:58 smithi082 ceph-mon[35046]: from='client.? 172.21.15.50:0/2557138502' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:37:58.854 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:37:58 smithi082 ceph-mon[35046]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:37:58.914 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:58 smithi050 systemd[1]: Stopping Ceph mon.a for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:37:59.290 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a[30374]: 2024-09-12T03:37:58.958+0000 7f52a0200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:37:59.290 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a[30374]: 2024-09-12T03:37:58.958+0000 7f52a0200640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:37:59.291 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:59 smithi050 podman[110729]: 2024-09-12 03:37:59.000652305 +0000 UTC m=+0.216114789 container died 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:37:59.656 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:59 smithi050 podman[110729]: 2024-09-12 03:37:59.422106854 +0000 UTC m=+0.637569325 container cleanup 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:37:59.657 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:59 smithi050 bash[110729]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:37:59.909 INFO:journalctl@ceph.mon.a.smithi050.stdout:Sep 12 03:37:59 smithi050 podman[110740]: 2024-09-12 03:37:59.656960817 +0000 UTC m=+0.640863403 container remove 606102499c9c5435e67ac22a68a9913e07cd06b7d0d1bcdbc1f500afa1fed094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:37:59.934 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:38:00.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:00.024 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:38:00.024 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:38:00.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:38:00.346 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:38:00 smithi082 systemd[1]: Stopping Ceph mon.b for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:00.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:38:00.110+0000 7fd993000640 -1 client.15162: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T03:38:00.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:38:00.111+0000 7fd993000640 -1 client.15162: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T03:38:00.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Exception in thread Thread-15: 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: Traceback (most recent call last): 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self.run() 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self.function(*self.args, **self.kwargs) 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: self.prune_snapshots(sched) 2024-09-12T03:38:00.415 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-12T03:38:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:38:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-b[35042]: 2024-09-12T03:38:00.344+0000 7f7bf021f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:38:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-b[35042]: 2024-09-12T03:38:00.344+0000 7f7bf021f640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:38:00.604 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:38:00 smithi082 podman[70506]: 2024-09-12 03:38:00.396596349 +0000 UTC m=+0.175012984 container died 21880aeea3edbceac4bcb2da5b80ebf5f7ed71b3d67ec2826e57a46c4cd76dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:38:01.055 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:38:00 smithi082 podman[70506]: 2024-09-12 03:38:00.804093589 +0000 UTC m=+0.582510227 container cleanup 21880aeea3edbceac4bcb2da5b80ebf5f7ed71b3d67ec2826e57a46c4cd76dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T03:38:01.055 INFO:journalctl@ceph.mon.b.smithi082.stdout:Sep 12 03:38:00 smithi082 bash[70506]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:38:01.188 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:38:01.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:01.236 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:38:01.236 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:38:01.236 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:38:01.543 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:38:01 smithi142 systemd[1]: Stopping Ceph mon.c for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:02.043 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:38:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c[35325]: 2024-09-12T03:38:01.571+0000 7f4a9914c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:02.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:38:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c[35325]: 2024-09-12T03:38:01.571+0000 7f4a9914c640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:38:02.044 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:38:01 smithi142 podman[72255]: 2024-09-12 03:38:01.796426421 +0000 UTC m=+0.363088737 container died 53642584f51a6524a05e7e7011444ea6de40227e455a878e66c89627277a392e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:38:02.465 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:38:02 smithi142 podman[72255]: 2024-09-12 03:38:02.197329028 +0000 UTC m=+0.763991341 container cleanup 53642584f51a6524a05e7e7011444ea6de40227e455a878e66c89627277a392e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T03:38:02.465 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 12 03:38:02 smithi142 bash[72255]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:38:02.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:38:02.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:02.665 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:38:02.665 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:38:02.665 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:38:03.080 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:02 smithi050 systemd[1]: Stopping Ceph mgr.x for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:03.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x[30603]: 2024-09-12T03:38:03.078+0000 7fda3c000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:03.414 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:03 smithi050 podman[110852]: 2024-09-12 03:38:03.163856602 +0000 UTC m=+0.203664974 container died d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:38:04.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:03 smithi050 podman[110852]: 2024-09-12 03:38:03.878031903 +0000 UTC m=+0.917840280 container cleanup d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T03:38:04.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:03 smithi050 bash[110852]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:38:04.164 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 12 03:38:04 smithi050 podman[110866]: 2024-09-12 03:38:04.071591727 +0000 UTC m=+0.896981224 container remove d8126bff4f32408ab991f88ee132a2093cdd1378b09df73bc0706cd4a3a26e22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:04.375 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:38:04.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:04.449 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:38:04.449 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:38:04.449 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:38:04.762 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:38:04 smithi082 systemd[1]: Stopping Ceph mgr.y for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:05.034 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:38:04 smithi082 podman[70629]: 2024-09-12 03:38:04.762786806 +0000 UTC m=+0.115839884 container died 349fd9d888ab90fc482fedd42a292b46fe2e9b2c384279ad5f49702c75689a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:38:05.354 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:38:05 smithi082 podman[70629]: 2024-09-12 03:38:05.134331487 +0000 UTC m=+0.487384610 container cleanup 349fd9d888ab90fc482fedd42a292b46fe2e9b2c384279ad5f49702c75689a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T03:38:05.354 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Sep 12 03:38:05 smithi082 bash[70629]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:38:05.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:38:05.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:05.607 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:38:05.608 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:38:05.608 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:38:05.956 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:38:05 smithi142 systemd[1]: Stopping Ceph mgr.z for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:06.253 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:38:05 smithi142 podman[72378]: 2024-09-12 03:38:05.956184733 +0000 UTC m=+0.152853298 container died cdaa4d58097445a53e497292b12ff2604a987c28b4ac1b4cc46ee2683e90e1aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:06.543 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:38:06 smithi142 podman[72378]: 2024-09-12 03:38:06.361065373 +0000 UTC m=+0.557733928 container cleanup cdaa4d58097445a53e497292b12ff2604a987c28b4ac1b4cc46ee2683e90e1aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:38:06.544 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 12 03:38:06 smithi142 bash[72378]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:38:06.777 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:38:06.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:06.811 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:38:06.811 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:38:06.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:38:07.164 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:06 smithi050 systemd[1]: Stopping Ceph osd.0 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:07.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0[44361]: 2024-09-12T03:38:07.235+0000 7f635fa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:07.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0[44361]: 2024-09-12T03:38:07.235+0000 7f635fa00640 -1 osd.0 108 *** Got signal Terminated *** 2024-09-12T03:38:07.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0[44361]: 2024-09-12T03:38:07.235+0000 7f635fa00640 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:12.967 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:12 smithi050 podman[110973]: 2024-09-12 03:38:12.705012493 +0000 UTC m=+5.599991513 container stop 59b19dc226f61457716c90dff70dd9dde79956f06213c6db0a4dfd64a13dd59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:38:12.968 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:12 smithi050 podman[110973]: 2024-09-12 03:38:12.730895945 +0000 UTC m=+5.625874963 container died 59b19dc226f61457716c90dff70dd9dde79956f06213c6db0a4dfd64a13dd59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:38:13.301 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:13 smithi050 podman[110973]: 2024-09-12 03:38:13.091624051 +0000 UTC m=+5.986603068 container cleanup 59b19dc226f61457716c90dff70dd9dde79956f06213c6db0a4dfd64a13dd59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:38:13.301 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:13 smithi050 bash[110973]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:38:13.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:13 smithi050 podman[110989]: 2024-09-12 03:38:13.301658533 +0000 UTC m=+0.582894053 container remove 59b19dc226f61457716c90dff70dd9dde79956f06213c6db0a4dfd64a13dd59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:38:14.139 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:13 smithi050 podman[111077]: 2024-09-12 03:38:13.771121374 +0000 UTC m=+0.027533786 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:38:14.139 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:13 smithi050 podman[111077]: 2024-09-12 03:38:13.894461953 +0000 UTC m=+0.150874347 container create 090c97a2cbfc04d3000e45f17342c576a709756f856baf9eebc3c3b8735b6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:38:14.139 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:14 smithi050 podman[111077]: 2024-09-12 03:38:14.139275329 +0000 UTC m=+0.395687751 container init 090c97a2cbfc04d3000e45f17342c576a709756f856baf9eebc3c3b8735b6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T03:38:14.410 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:14 smithi050 podman[111077]: 2024-09-12 03:38:14.146459393 +0000 UTC m=+0.402871787 container start 090c97a2cbfc04d3000e45f17342c576a709756f856baf9eebc3c3b8735b6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:38:14.410 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:14 smithi050 podman[111077]: 2024-09-12 03:38:14.247245093 +0000 UTC m=+0.503657503 container attach 090c97a2cbfc04d3000e45f17342c576a709756f856baf9eebc3c3b8735b6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:38:14.664 INFO:journalctl@ceph.osd.0.smithi050.stdout:Sep 12 03:38:14 smithi050 podman[111077]: 2024-09-12 03:38:14.410504606 +0000 UTC m=+0.666917014 container died 090c97a2cbfc04d3000e45f17342c576a709756f856baf9eebc3c3b8735b6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T03:38:15.091 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:38:15.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:15.168 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:38:15.168 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:38:15.168 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:38:15.354 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:15 smithi082 systemd[1]: Stopping Ceph osd.1 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:15.854 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1[39517]: 2024-09-12T03:38:15.471+0000 7f8ac970d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:15.854 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1[39517]: 2024-09-12T03:38:15.471+0000 7f8ac970d640 -1 osd.1 108 *** Got signal Terminated *** 2024-09-12T03:38:15.854 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1[39517]: 2024-09-12T03:38:15.471+0000 7f8ac970d640 -1 osd.1 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:21.263 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:20 smithi082 podman[70763]: 2024-09-12 03:38:20.969226921 +0000 UTC m=+5.615618915 container died f049cd9c18a088bec3c46c2b6fe669cab403039b0075d2b14096e013e8ae15f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:38:21.604 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:21 smithi082 podman[70763]: 2024-09-12 03:38:21.354419681 +0000 UTC m=+6.000811669 container cleanup f049cd9c18a088bec3c46c2b6fe669cab403039b0075d2b14096e013e8ae15f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T03:38:21.604 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:21 smithi082 bash[70763]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:38:22.016 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:21 smithi082 podman[70781]: 2024-09-12 03:38:21.656062709 +0000 UTC m=+0.688490487 container remove f049cd9c18a088bec3c46c2b6fe669cab403039b0075d2b14096e013e8ae15f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:22.342 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:22 smithi082 podman[70868]: 2024-09-12 03:38:21.919785037 +0000 UTC m=+0.015438751 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:38:22.342 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:22 smithi082 podman[70868]: 2024-09-12 03:38:22.031920541 +0000 UTC m=+0.127574288 container create d10f52a5a92e6c49d82b66690bde1e29c409266fc674011fbbd782d514b9813c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:38:22.342 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:22 smithi082 podman[70868]: 2024-09-12 03:38:22.251011163 +0000 UTC m=+0.346664873 container init d10f52a5a92e6c49d82b66690bde1e29c409266fc674011fbbd782d514b9813c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:38:22.342 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:22 smithi082 podman[70868]: 2024-09-12 03:38:22.254472181 +0000 UTC m=+0.350125891 container start d10f52a5a92e6c49d82b66690bde1e29c409266fc674011fbbd782d514b9813c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:22.342 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:22 smithi082 podman[70868]: 2024-09-12 03:38:22.342670905 +0000 UTC m=+0.438324613 container attach d10f52a5a92e6c49d82b66690bde1e29c409266fc674011fbbd782d514b9813c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:38:22.604 INFO:journalctl@ceph.osd.1.smithi082.stdout:Sep 12 03:38:22 smithi082 podman[70868]: 2024-09-12 03:38:22.439715817 +0000 UTC m=+0.535369530 container died d10f52a5a92e6c49d82b66690bde1e29c409266fc674011fbbd782d514b9813c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:38:23.064 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:38:23.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:23.105 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:38:23.105 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:38:23.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:38:23.424 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:23 smithi142 systemd[1]: Stopping Ceph osd.2 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:23.793 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2[40268]: 2024-09-12T03:38:23.423+0000 7f631889e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:23.794 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2[40268]: 2024-09-12T03:38:23.423+0000 7f631889e640 -1 osd.2 108 *** Got signal Terminated *** 2024-09-12T03:38:23.794 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2[40268]: 2024-09-12T03:38:23.423+0000 7f631889e640 -1 osd.2 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:29.175 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:28 smithi142 podman[72511]: 2024-09-12 03:38:28.879423185 +0000 UTC m=+5.576567293 container died cdbb3bd5d1ff14c9ff0d8ec2841c3267668365b9643da14ee1c9a48540b83768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:38:29.483 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:29 smithi142 podman[72511]: 2024-09-12 03:38:29.273985009 +0000 UTC m=+5.971129116 container cleanup cdbb3bd5d1ff14c9ff0d8ec2841c3267668365b9643da14ee1c9a48540b83768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:29.483 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:29 smithi142 bash[72511]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:38:29.793 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:29 smithi142 podman[72527]: 2024-09-12 03:38:29.48385567 +0000 UTC m=+0.601655903 container remove cdbb3bd5d1ff14c9ff0d8ec2841c3267668365b9643da14ee1c9a48540b83768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:38:30.170 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:29 smithi142 podman[72618]: 2024-09-12 03:38:29.792986062 +0000 UTC m=+0.016185620 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:38:30.170 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:29 smithi142 podman[72618]: 2024-09-12 03:38:29.909987135 +0000 UTC m=+0.133186679 container create 1dfed8cc8b76406d90369e22c68399d29678eec66e20c241f76dca5085c09909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T03:38:30.544 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:30 smithi142 podman[72618]: 2024-09-12 03:38:30.171182961 +0000 UTC m=+0.394382505 container init 1dfed8cc8b76406d90369e22c68399d29678eec66e20c241f76dca5085c09909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:38:30.544 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:30 smithi142 podman[72618]: 2024-09-12 03:38:30.174889143 +0000 UTC m=+0.398088690 container start 1dfed8cc8b76406d90369e22c68399d29678eec66e20c241f76dca5085c09909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:38:30.544 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:30 smithi142 podman[72618]: 2024-09-12 03:38:30.296373751 +0000 UTC m=+0.519573302 container attach 1dfed8cc8b76406d90369e22c68399d29678eec66e20c241f76dca5085c09909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:38:30.544 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 12 03:38:30 smithi142 podman[72618]: 2024-09-12 03:38:30.366148591 +0000 UTC m=+0.589348136 container died 1dfed8cc8b76406d90369e22c68399d29678eec66e20c241f76dca5085c09909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:31.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:38:31.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:31.083 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:38:31.083 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:38:31.083 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:38:31.414 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:31 smithi050 systemd[1]: Stopping Ceph osd.3 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:31.746 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:31 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3[50827]: 2024-09-12T03:38:31.523+0000 7f786b600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:31.746 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:31 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3[50827]: 2024-09-12T03:38:31.523+0000 7f786b600640 -1 osd.3 108 *** Got signal Terminated *** 2024-09-12T03:38:31.746 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:31 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3[50827]: 2024-09-12T03:38:31.523+0000 7f786b600640 -1 osd.3 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:33.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:33.209+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:33.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:33.422+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:34.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:34 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:34.172+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:34.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:34.391+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:34.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:34.347+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:35.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:35 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:35.157+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:35.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:35.372+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:35.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:35.355+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:35.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:35.345+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:35.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:35 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:35.614+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:36.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:36 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:36.165+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:36.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:36.341+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:36.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:36.368+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:36.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:36.367+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:36.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:36 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:36.657+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:37.405 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:37 smithi050 podman[111179]: 2024-09-12 03:38:37.098545612 +0000 UTC m=+5.716861634 container died 1e432eb2b0fa6aa0b3c0f9690354f95f5c1fe17e7973de72912a141f602517b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:38:37.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:37 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:37.211+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:37.664 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:37 smithi050 podman[111179]: 2024-09-12 03:38:37.546440299 +0000 UTC m=+6.164756320 container cleanup 1e432eb2b0fa6aa0b3c0f9690354f95f5c1fe17e7973de72912a141f602517b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:37.664 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:37 smithi050 bash[111179]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:38:37.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:37.336+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:37.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:37.405+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:37.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:37.336+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:38.164 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:37 smithi050 podman[111211]: 2024-09-12 03:38:37.789783735 +0000 UTC m=+0.675996024 container remove 1e432eb2b0fa6aa0b3c0f9690354f95f5c1fe17e7973de72912a141f602517b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T03:38:38.165 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:37 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:37.693+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:38.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:38 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:38.257+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:38.652 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:38 smithi050 podman[111299]: 2024-09-12 03:38:38.270948652 +0000 UTC m=+0.027220760 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:38:38.652 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:38 smithi050 podman[111299]: 2024-09-12 03:38:38.399096639 +0000 UTC m=+0.155368724 container create 0b567fe107cb0dbf113b800c2cb2a1d1d9a5391bf5edef58a105feec5acd0c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:38:38.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:38.340+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:38.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:38.353+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:38.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:38.392+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:38.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:38 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:38.670+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:38.916 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:38 smithi050 podman[111299]: 2024-09-12 03:38:38.652213584 +0000 UTC m=+0.408485671 container init 0b567fe107cb0dbf113b800c2cb2a1d1d9a5391bf5edef58a105feec5acd0c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:38:38.917 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:38 smithi050 podman[111299]: 2024-09-12 03:38:38.659554165 +0000 UTC m=+0.415826259 container start 0b567fe107cb0dbf113b800c2cb2a1d1d9a5391bf5edef58a105feec5acd0c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:38:38.917 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:38 smithi050 podman[111299]: 2024-09-12 03:38:38.76021068 +0000 UTC m=+0.516482774 container attach 0b567fe107cb0dbf113b800c2cb2a1d1d9a5391bf5edef58a105feec5acd0c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T03:38:39.254 INFO:journalctl@ceph.osd.3.smithi050.stdout:Sep 12 03:38:38 smithi050 podman[111299]: 2024-09-12 03:38:38.934149387 +0000 UTC m=+0.690421473 container died 0b567fe107cb0dbf113b800c2cb2a1d1d9a5391bf5edef58a105feec5acd0c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:38:39.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:38:39.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:39 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:39.289+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:39.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:39.648 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:38:39.648 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:38:39.648 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:38:39.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:39.432+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:39.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:39.395+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:39.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:39.299+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:39.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:39 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:39.664+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:39.964 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:39 smithi082 systemd[1]: Stopping Ceph osd.4 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:40.240 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:39 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:38:39.964+0000 7f293009f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:40.240 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:39 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:38:39.964+0000 7f293009f640 -1 osd.4 108 *** Got signal Terminated *** 2024-09-12T03:38:40.240 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:39 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:38:39.964+0000 7f293009f640 -1 osd.4 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:40.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:40 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:40.239+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:40.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:40.331+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:40.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:40.411+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:40.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:40.428+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:40.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:40 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:40.629+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:41.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:41 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:41.207+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:41.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:41 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:41.207+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:41.764 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:41.388+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:41.765 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:41.362+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:41.765 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:41.339+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:41.765 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:41.339+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:42.164 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:41 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:41.670+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:42.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:42 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:42.206+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:42.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:42 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:42.206+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:42.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:42 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:42.384+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:42.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:42 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:42.336+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:42.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:42 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:42.336+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:42.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:42 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:42.428+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:42.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:42 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:42.428+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:42.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:42 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:42.640+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:43.505 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:43 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:43.250+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:43.505 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:43 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:43.242+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:43.506 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:43 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:43.242+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:43.696 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:43 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:43.425+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:43.696 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:43 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:43.425+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:43.696 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:43 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:43.354+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:43.697 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:43 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:43.354+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:43.697 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:43 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:43.330+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:43.697 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:43 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:43.330+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:43.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:43 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:43.626+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:44.511 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:44 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:44.225+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:44.511 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:44 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:44.218+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:44.511 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:44 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:44.218+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:44.786 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:44 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4[45949]: 2024-09-12T03:38:44.511+0000 7f292c6b5640 -1 osd.4 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:16.607240+0000 front 2024-09-12T03:38:16.607264+0000 (oldest deadline 2024-09-12T03:38:43.608158+0000) 2024-09-12T03:38:44.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:44 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:44.309+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:44.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:44 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:44.309+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:44.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:44 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:44.442+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:44.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:44 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:44.442+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:44.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:44 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:44.398+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:44.795 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:44 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:44.398+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:44.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:44.658+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:45.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:44.965+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:45.604 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:45 smithi082 podman[70970]: 2024-09-12 03:38:45.339522885 +0000 UTC m=+5.496726466 container died cadf6e150936a3fb6b165544ae4aeffded51320eddb590aa20706df50204b60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:38:45.605 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:45 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:45.188+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:45.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:45 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:45.222+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:45.606 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:45 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:45.222+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:45.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:45 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:45.295+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:45.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:45 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:45.295+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:45.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:45 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:45.391+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:45.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:45 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:45.391+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:45.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:45 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:45.437+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:45.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:45 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:45.437+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:45.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:45 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:45.624+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:45.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:45 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:45.624+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:45.941 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:45 smithi082 podman[70970]: 2024-09-12 03:38:45.722747599 +0000 UTC m=+5.879951169 container cleanup cadf6e150936a3fb6b165544ae4aeffded51320eddb590aa20706df50204b60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T03:38:45.941 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:45 smithi082 bash[70970]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:38:46.194 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:45 smithi082 podman[70996]: 2024-09-12 03:38:45.940908606 +0000 UTC m=+0.601003982 container remove cadf6e150936a3fb6b165544ae4aeffded51320eddb590aa20706df50204b60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T03:38:46.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:45 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:45.925+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:46.488 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:46 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:46.193+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:46.489 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:46 smithi082 podman[71085]: 2024-09-12 03:38:46.277702417 +0000 UTC m=+0.015792673 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:38:46.489 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:46 smithi082 podman[71085]: 2024-09-12 03:38:46.400276998 +0000 UTC m=+0.138367245 container create 4807b183bab0c33f01dee44081c904f74fa1e85282466047c114abc9a24b860a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:38:46.489 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:46 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:46.251+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:46.489 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:46 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:46.251+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:46.769 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:46 smithi082 podman[71085]: 2024-09-12 03:38:46.577325951 +0000 UTC m=+0.315416197 container init 4807b183bab0c33f01dee44081c904f74fa1e85282466047c114abc9a24b860a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:38:46.770 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:46 smithi082 podman[71085]: 2024-09-12 03:38:46.580632178 +0000 UTC m=+0.318722432 container start 4807b183bab0c33f01dee44081c904f74fa1e85282466047c114abc9a24b860a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T03:38:46.770 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:46 smithi082 podman[71085]: 2024-09-12 03:38:46.660808745 +0000 UTC m=+0.398899017 container attach 4807b183bab0c33f01dee44081c904f74fa1e85282466047c114abc9a24b860a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:38:46.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:46.302+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:46.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:46.302+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:46.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:46.435+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:46.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:46.435+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:46.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:46.484+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:46.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:46 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:46.484+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:46.884 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:46 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:46.592+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:46.885 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:46 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:46.592+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:47.104 INFO:journalctl@ceph.osd.4.smithi082.stdout:Sep 12 03:38:46 smithi082 podman[71085]: 2024-09-12 03:38:46.770256474 +0000 UTC m=+0.508346721 container died 4807b183bab0c33f01dee44081c904f74fa1e85282466047c114abc9a24b860a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:38:47.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:46 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:46.884+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:47.414 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:47 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:47.181+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:47.414 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:47 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:47.287+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:47.415 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:47 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:47.287+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:47.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:38:47.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:47.475 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:38:47.476 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:38:47.476 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:38:47.530 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:47.279+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:47.530 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:47.279+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:47.531 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:47.497+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:47.531 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:47.497+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:47.531 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:47.436+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:47.531 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:47.436+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:47.793 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:47 smithi142 systemd[1]: Stopping Ceph osd.5 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:47.896 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:47 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:47.565+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:47.897 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:47 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:47.565+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:48.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:47 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:47.896+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:48.274 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:47.794+0000 7fe6ba0a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:48.274 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:47.794+0000 7fe6ba0a7640 -1 osd.5 108 *** Got signal Terminated *** 2024-09-12T03:38:48.274 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:47 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:47.794+0000 7fe6ba0a7640 -1 osd.5 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:48.274 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:48 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:48.273+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:48.274 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:48 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:48.273+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:48.532 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:48 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:48.388+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:48.532 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:48 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:48.388+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:48.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:48 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:48.140+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:48.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:48 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:48.249+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:48.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:48 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:48.249+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:48.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:48 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:48.531+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:48.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:48 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:48.531+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:48.904 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:48.605+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:48.904 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:48.605+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:49.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:48 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:48.904+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:49.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:49.185+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:49.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:49.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:49.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:49 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:49.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:49.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:49.499+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:49.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:49.499+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:49.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:49.312+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:49.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:49.312+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:49.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:49.375+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:49.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:49.375+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:49.795 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:49 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:49.375+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:49.897 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:49 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:49.647+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:49.898 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:49 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:49.647+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:50.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:49 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:49.898+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:50.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:49 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:49.898+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:50.543 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:50.279+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:50.543 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:50.279+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:50.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:50.517+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:50.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:50.517+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:50.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:50.517+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:50.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:50.326+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:50.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:50.326+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:50.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:50 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:50.326+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:50.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:50 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:50.158+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:50.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:50 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:50.158+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:50.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:50 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:50.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:50.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:50 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:50.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:50.905 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:50.597+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:50.905 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:50.597+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:51.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:50.905+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:51.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:50.905+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:51.543 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:51.289+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:51.543 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:51.289+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:51.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:51.480+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:51.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:51.480+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:51.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:51.480+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:51.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:51.298+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:51.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:51.298+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:51.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:51 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:51.298+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:51.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:51 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:51.119+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:51.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:51 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:51.119+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:51.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:51 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:51.214+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:51.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:51 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:51.214+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:51.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:51 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:51.566+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:51.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:51 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:51.566+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:51.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:51 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:51.566+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:52.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:52 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:52.070+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:52.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:52 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:52.070+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:52.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:52 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:52.166+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:52.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:52 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:52.166+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:52.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:51 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:51.939+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:52.416 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:51 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:51.939+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:52.543 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:52.306+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.550239+0000 front 2024-09-12T03:38:10.550255+0000 (oldest deadline 2024-09-12T03:38:35.250308+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5[47192]: 2024-09-12T03:38:52.306+0000 7fe6b66bd640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.151284+0000 front 2024-09-12T03:38:18.151370+0000 (oldest deadline 2024-09-12T03:38:41.051258+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:52.472+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:52.472+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:52.472+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:52.290+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:52.290+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:52.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:52 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:52.290+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:52.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:52 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:52.594+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:52.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:52 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:52.594+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:52.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:52 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:52.594+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:53.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:52 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:52.981+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:53.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:52 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:52.981+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:53.509 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:53 smithi142 podman[72734]: 2024-09-12 03:38:53.22088128 +0000 UTC m=+5.543164964 container died 7da7e3ad2a3ad4b7a1637fd95d95e180a73547aa8f26d690b917c966cf15cb58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T03:38:53.509 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:53 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:53.262+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:53.510 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:53 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:53.262+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:53.510 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:53 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:53.262+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:53.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:53 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:53.117+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:53.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:53 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:53.117+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:53.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:53 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:53.128+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:53.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:53 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:53.128+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:53.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:53 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:53.128+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:53.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:53 smithi142 podman[72734]: 2024-09-12 03:38:53.636263931 +0000 UTC m=+5.958547602 container cleanup 7da7e3ad2a3ad4b7a1637fd95d95e180a73547aa8f26d690b917c966cf15cb58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T03:38:53.794 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:53 smithi142 bash[72734]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:38:53.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:53 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:53.509+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:53.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:53 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:53.509+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:53.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:53 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:53.509+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:53.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:53 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:53.567+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:53.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:53 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:53.567+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:53.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:53 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:53.567+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:54.293 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:53 smithi142 podman[72750]: 2024-09-12 03:38:53.896284197 +0000 UTC m=+0.674927810 container remove 7da7e3ad2a3ad4b7a1637fd95d95e180a73547aa8f26d690b917c966cf15cb58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:38:54.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:54 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:54.087+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:54.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:54 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:54.087+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:54.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:54 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:54.091+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:54.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:54 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:54.091+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:54.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:54 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:54.091+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:54.415 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:54.027+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:54.415 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:54.027+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:54.557 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:54 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:54.305+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:54 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:54.305+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:54 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:54.305+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:54 smithi142 podman[72840]: 2024-09-12 03:38:54.297271423 +0000 UTC m=+0.108204696 container create d1a1d6f889d80dfb9bc7819dc9903f6d0f9c4c3b9a63f41142af4a10432fa32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:54 smithi142 podman[72840]: 2024-09-12 03:38:54.20489676 +0000 UTC m=+0.015829999 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:54 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:54.492+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:54 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:54.492+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:54.558 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:54 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:54.492+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:54.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:54.525+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:54.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:54.525+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:54.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:54.525+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:55.044 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:54 smithi142 podman[72840]: 2024-09-12 03:38:54.558165151 +0000 UTC m=+0.369098382 container init d1a1d6f889d80dfb9bc7819dc9903f6d0f9c4c3b9a63f41142af4a10432fa32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:38:55.044 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:54 smithi142 podman[72840]: 2024-09-12 03:38:54.561626268 +0000 UTC m=+0.372559501 container start d1a1d6f889d80dfb9bc7819dc9903f6d0f9c4c3b9a63f41142af4a10432fa32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:38:55.044 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:54 smithi142 podman[72840]: 2024-09-12 03:38:54.66671515 +0000 UTC m=+0.477648385 container attach d1a1d6f889d80dfb9bc7819dc9903f6d0f9c4c3b9a63f41142af4a10432fa32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:38:55.044 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 12 03:38:54 smithi142 podman[72840]: 2024-09-12 03:38:54.7441228 +0000 UTC m=+0.555056033 container died d1a1d6f889d80dfb9bc7819dc9903f6d0f9c4c3b9a63f41142af4a10432fa32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:38:55.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:55 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:55.129+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:55.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:55 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:55.129+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:55.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:55 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:55.081+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:55.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:55 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:55.081+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:55.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:55 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:55.081+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:55.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:55.012+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:55.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:55.012+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:55.428 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:55 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:55.293+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:55.428 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:55 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:55.293+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:55.429 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:55 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:55.293+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:55.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:38:55.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:38:55.497 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:38:55.497 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:38:55.497 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:38:55.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:55 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:55.465+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:55.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:55 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:55.465+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:55.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:55 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:55.465+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:55.914 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:55 smithi050 systemd[1]: Stopping Ceph osd.6 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:38:55.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:55.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:55.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:55.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:55.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:55.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:56.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:56 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:56.141+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:56.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:56 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:56.141+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:56.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:56 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:56.086+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:56.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:56 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:56.086+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:56.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:56 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:56.086+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:56.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:55.948+0000 7f6cad600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:38:56.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:55.948+0000 7f6cad600640 -1 osd.6 108 *** Got signal Terminated *** 2024-09-12T03:38:56.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:55 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:55.948+0000 7f6cad600640 -1 osd.6 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:38:56.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:56 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:56.041+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:56.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:56 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:56.041+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:56.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:56 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:56.417+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:56.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:56 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:56.417+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:56.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:56 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:56.417+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:56.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:56 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:56.320+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:56.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:56 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:56.320+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:56.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:56 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:56.320+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:56.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:56 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:56.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:56.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:56 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:56.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:56.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:56 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:56.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:57.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:57 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:57.157+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:57.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:57 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:57.157+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:57.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:57 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:57.052+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:57.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:57 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:57.052+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:57.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:57 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:57.052+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:57.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:57 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:57.068+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:57.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:57 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:57.069+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:57.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:57 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:57.439+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:57.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:57 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:57.439+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:57.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:57 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:57.439+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:57.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:57 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:57.292+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:57.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:57 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:57.292+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:57.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:57 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:57.292+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:57.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:57 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:57.431+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:57.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:57 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:57.431+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:57.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:57 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:57.431+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:58.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:58.149+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:58.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:58.149+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:58.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:58.149+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:38:58.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:58.099+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:58.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:58.099+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:58.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:58.099+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:58.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:58.099+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:38:58.396 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:58.078+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:58.396 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:58.078+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:58.396 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:58.078+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.904654+0000 front 2024-09-12T03:38:33.904630+0000 (oldest deadline 2024-09-12T03:38:57.404782+0000) 2024-09-12T03:38:58.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:58 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:58.459+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:58.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:58 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:58.459+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:58.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:58 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:58.459+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:58.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:58 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:58.270+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:58.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:58 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:58.270+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:58.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:58 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:58.270+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:58.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:58.396+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:58.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:58.396+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:58.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:58 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:58.396+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:59.413 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:59.107+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:38:59.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:59.107+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:38:59.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:38:59.107+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.904654+0000 front 2024-09-12T03:38:33.904630+0000 (oldest deadline 2024-09-12T03:38:57.404782+0000) 2024-09-12T03:38:59.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:59.118+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:38:59.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:59.118+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:38:59.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:38:59.118+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:38:59.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:59.107+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:38:59.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:59.107+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:38:59.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:59.107+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:38:59.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:38:59 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:38:59.107+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:38:59.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:59.414+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:38:59.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:59.414+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:38:59.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:59.414+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:38:59.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:38:59 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:38:59.414+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:38:59.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:59.309+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:38:59.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:59.309+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:38:59.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:38:59.309+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:38:59.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:59.468+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:38:59.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:59.468+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:38:59.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:59.468+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:38:59.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:38:59 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:38:59.468+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:00.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:39:00.152+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:14.601030+0000 front 2024-09-12T03:38:14.601185+0000 (oldest deadline 2024-09-12T03:38:44.601953+0000) 2024-09-12T03:39:00.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:39:00.152+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.503476+0000 front 2024-09-12T03:38:27.503602+0000 (oldest deadline 2024-09-12T03:38:49.803210+0000) 2024-09-12T03:39:00.414 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6[57617]: 2024-09-12T03:39:00.152+0000 7f6ca4000640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.904654+0000 front 2024-09-12T03:38:33.904630+0000 (oldest deadline 2024-09-12T03:38:57.404782+0000) 2024-09-12T03:39:00.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:00.116+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:00.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:00.116+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:00.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:00.116+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:00.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:00.154+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:00.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:00.154+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:00.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:00.154+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:00.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:00 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:00.155+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:00.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:00.294+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:00.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:00.294+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:00.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:00.294+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:00.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:00.454+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:00.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:00.454+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:00.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:00.454+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:00.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:00 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:00.454+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:00.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:00.416+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:00.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:00.416+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:00.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:00.416+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:00.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:00 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:00.416+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:01.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:01.111+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:01.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:01.111+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:01.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:01.111+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:01.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:01.121+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:01.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:01.121+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:01.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:01.121+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:01.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:01 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:01.121+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:01.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:01.307+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:01.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:01.307+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:01.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:01.307+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:01.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:01.406+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:01.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:01.406+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:01.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:01.406+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:01.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:01 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:01.406+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:01.802 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:01 smithi050 podman[111398]: 2024-09-12 03:39:01.506003383 +0000 UTC m=+5.722821428 container died 0b94aa457bf67e9e447ebe42a47b11fd41404ef785f0e9f501a5433c511892dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:39:01.803 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:01 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:01.437+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:01.803 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:01 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:01.437+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:01.803 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:01 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:01.437+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:01.803 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:01 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:01.437+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:02.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:01 smithi050 podman[111398]: 2024-09-12 03:39:01.917201745 +0000 UTC m=+6.134019786 container cleanup 0b94aa457bf67e9e447ebe42a47b11fd41404ef785f0e9f501a5433c511892dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:39:02.164 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:01 smithi050 bash[111398]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:39:02.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:02.095+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:02.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:02.095+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:02.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:02.095+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:02.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:02.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:02.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:02.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:02.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:02.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:02.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:02.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:02.664 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:02 smithi050 podman[111415]: 2024-09-12 03:39:02.177135378 +0000 UTC m=+0.667268610 container remove 0b94aa457bf67e9e447ebe42a47b11fd41404ef785f0e9f501a5433c511892dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:39:02.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:02 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:02.412+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:02.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:02 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:02.412+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:02.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:02 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:02.412+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:02.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:02 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:02.412+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:02.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:02.294+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:02.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:02.294+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:02.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:02.294+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:02.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:02.401+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:02.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:02.401+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:02.795 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:02.401+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:02.795 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:02 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:02.401+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:02.997 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:02 smithi050 podman[111501]: 2024-09-12 03:39:02.642582304 +0000 UTC m=+0.027479196 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:39:02.997 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:02 smithi050 podman[111501]: 2024-09-12 03:39:02.761607068 +0000 UTC m=+0.146503940 container create ea3212f8fc9ffdba7f86bcc1e91f84c2f154d3edf001865c4dde77fefb0c5d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:39:02.998 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:02 smithi050 podman[111501]: 2024-09-12 03:39:02.997971931 +0000 UTC m=+0.382868825 container init ea3212f8fc9ffdba7f86bcc1e91f84c2f154d3edf001865c4dde77fefb0c5d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:39:03.266 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:03 smithi050 podman[111501]: 2024-09-12 03:39:03.005004324 +0000 UTC m=+0.389901196 container start ea3212f8fc9ffdba7f86bcc1e91f84c2f154d3edf001865c4dde77fefb0c5d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:39:03.266 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:03 smithi050 podman[111501]: 2024-09-12 03:39:03.097480934 +0000 UTC m=+0.482377819 container attach ea3212f8fc9ffdba7f86bcc1e91f84c2f154d3edf001865c4dde77fefb0c5d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:39:03.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:03.061+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:03.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:03.061+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:03.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:03.061+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:03.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:03.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:03.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:03.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:03.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:03.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:03.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:03 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:03.116+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:03.253+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:03.253+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:03.253+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:03.399+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:03.399+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:03.399+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:03.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:03 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:03.399+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:03.620 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 12 03:39:03 smithi050 podman[111524]: 2024-09-12 03:39:03.306594558 +0000 UTC m=+0.026499158 container died ea3212f8fc9ffdba7f86bcc1e91f84c2f154d3edf001865c4dde77fefb0c5d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:39:03.620 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:03.420+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:03.620 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:03.420+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:03.621 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:03.420+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:03.621 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:03 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:03.420+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:03.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:39:04.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:04.043 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:39:04.043 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:39:04.043 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:39:04.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:04.063+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:04.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:04.063+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:04.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:04.063+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:04.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 systemd[1]: Stopping Ceph osd.7 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:04.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:04.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:04.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:04.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:04.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:04.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:04.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:04.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:04.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:04.286+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:04.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:04.286+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:04.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:04.286+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:04.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:04.362+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:04.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:04.362+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:04.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:04.362+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:04.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:04 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:04.362+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:04.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:04 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:04.407+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:04.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:04 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:04.407+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:04.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:04 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:04.407+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:04.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:04 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:04.407+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:04.854 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:04.374+0000 7fa000f24640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:04.854 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:04.374+0000 7fa000f24640 -1 osd.7 108 *** Got signal Terminated *** 2024-09-12T03:39:04.854 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:04 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:04.374+0000 7fa000f24640 -1 osd.7 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:39:05.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:05.036+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:05.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:05.036+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:05.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:05.036+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:05.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:05.149+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:05.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:05.149+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:05.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:05.149+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:05.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:05 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:05.149+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:05.793 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:05.317+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:05.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:05.317+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:05.794 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:05.317+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:05.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:05.333+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:05.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:05.333+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:05.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:05.333+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:05.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:05 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:05.333+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:05.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:05.439+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:05.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:05.439+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:05.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:05.439+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:05.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:05 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:05.439+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:06.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:06.194+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:06.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:06.194+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:06.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:06.194+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:06.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:06.194+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:06.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:06.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:06.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:06.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:06.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:06.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:06.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:06.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:41.267468+0000 front 2024-09-12T03:38:41.267530+0000 (oldest deadline 2024-09-12T03:39:05.367550+0000) 2024-09-12T03:39:06.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:06.291+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:06.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:06.291+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:06.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:06.291+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:06.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:06.303+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:06.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:06.303+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:06.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:06.303+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:06.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:06 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:06.303+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:06.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:06.424+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:06.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:06.424+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:06.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:06.424+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:06.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:06 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:06.424+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:07.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:07.156+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:07.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:07.156+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:07.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:07.156+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:07.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:07.156+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:07.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:07.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:07.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:07.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:07.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:07.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:07.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:07 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:07.031+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:41.267468+0000 front 2024-09-12T03:38:41.267530+0000 (oldest deadline 2024-09-12T03:39:05.367550+0000) 2024-09-12T03:39:07.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:07.248+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:07.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:07.248+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:07.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:07.248+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:07.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:07.265+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:07.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:07.265+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:07.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:07.265+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:07.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:07 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:07.265+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:07.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:07.436+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:07.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:07.436+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:07.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:07.436+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:07.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:07 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:07.436+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:08.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:08.200+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:08.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:08.200+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:08.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:08.200+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:08.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:08.200+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:08.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:08.200+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:08.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:08.042+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:08.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:08.042+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:08.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:08.042+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:08.355 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:08 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:08.042+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:41.267468+0000 front 2024-09-12T03:38:41.267530+0000 (oldest deadline 2024-09-12T03:39:05.367550+0000) 2024-09-12T03:39:08.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:08.252+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:08.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:08.252+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:08.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:08.252+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:08.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:08.278+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:08.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:08.278+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:08.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:08.278+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:08.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:08 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:08.278+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:08.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:08.476+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:08.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:08.476+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:08.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:08.476+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:08.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:08 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:08.476+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:09.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:09.048+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:17.162145+0000 front 2024-09-12T03:38:17.162172+0000 (oldest deadline 2024-09-12T03:38:43.062341+0000) 2024-09-12T03:39:09.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:09.048+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:28.164039+0000 front 2024-09-12T03:38:28.164091+0000 (oldest deadline 2024-09-12T03:38:49.864136+0000) 2024-09-12T03:39:09.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:09.048+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.265740+0000 front 2024-09-12T03:38:33.265763+0000 (oldest deadline 2024-09-12T03:38:57.866303+0000) 2024-09-12T03:39:09.354 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7[52696]: 2024-09-12T03:39:09.048+0000 7f9ff9532640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:41.267468+0000 front 2024-09-12T03:38:41.267530+0000 (oldest deadline 2024-09-12T03:39:05.367550+0000) 2024-09-12T03:39:09.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:09.195+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:09.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:09.195+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:09.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:09.195+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:09.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:09.195+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:09.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:09 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:09.195+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:09.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:09.316+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:09.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:09.316+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:09.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:09.316+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:09.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:09.316+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:09.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:09.255+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:09.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:09.255+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:09.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:09.255+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:09.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:09 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:09.255+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:09.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:09.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:09.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:09.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:09.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:09.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:09.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:09.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:09.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:09 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:09.465+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:10.149 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:09 smithi082 podman[71189]: 2024-09-12 03:39:09.868809781 +0000 UTC m=+5.631147472 container died 6da97a328ab3f5b81468160f385be274563d86290bd6a84fa0ee58062390bd93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:10 smithi082 podman[71189]: 2024-09-12 03:39:10.257488165 +0000 UTC m=+6.019825846 container cleanup 6da97a328ab3f5b81468160f385be274563d86290bd6a84fa0ee58062390bd93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:10 smithi082 bash[71189]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:10.227+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:10.227+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:10.227+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:10.227+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:10.525 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:10.227+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:10.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:10.353+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:10.353+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:10.353+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:10.353+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:10.244+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:10.244+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:10.244+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:10.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:10 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:10.244+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:10.854 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:10 smithi082 podman[71204]: 2024-09-12 03:39:10.525477455 +0000 UTC m=+0.646230229 container remove 6da97a328ab3f5b81468160f385be274563d86290bd6a84fa0ee58062390bd93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:39:10.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:10.487+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:10.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:10.487+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:10.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:10.487+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:10.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:10.487+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:10.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:10 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:10.487+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:11.153 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:10 smithi082 podman[71291]: 2024-09-12 03:39:10.774927419 +0000 UTC m=+0.015465029 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:39:11.153 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:10 smithi082 podman[71291]: 2024-09-12 03:39:10.884686941 +0000 UTC m=+0.125224596 container create 21a18a64b793c64a538d3b2a553c828194dad31ee1f844a3ffa51645da837ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:39:11.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:11.232+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:11.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:11.232+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:11.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:11.232+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:11.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:11.232+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:11.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:11.308+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:11.546 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:11.308+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:11.546 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:11.308+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:11.546 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:11 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:11.308+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:11.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:11 smithi082 podman[71291]: 2024-09-12 03:39:11.153829729 +0000 UTC m=+0.394367350 container init 21a18a64b793c64a538d3b2a553c828194dad31ee1f844a3ffa51645da837ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:39:11.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:11 smithi082 podman[71291]: 2024-09-12 03:39:11.157669939 +0000 UTC m=+0.398207545 container start 21a18a64b793c64a538d3b2a553c828194dad31ee1f844a3ffa51645da837ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:39:11.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:11 smithi082 podman[71291]: 2024-09-12 03:39:11.245315199 +0000 UTC m=+0.485852830 container attach 21a18a64b793c64a538d3b2a553c828194dad31ee1f844a3ffa51645da837ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:39:11.604 INFO:journalctl@ceph.osd.7.smithi082.stdout:Sep 12 03:39:11 smithi082 podman[71314]: 2024-09-12 03:39:11.369925889 +0000 UTC m=+0.016065623 container died 21a18a64b793c64a538d3b2a553c828194dad31ee1f844a3ffa51645da837ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:39:11.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:11 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:11.191+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:11.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:11 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:11.191+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:11.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:11 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:11.191+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:11.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:11 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:11.191+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:11.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:11 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:11.191+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:11.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:11.457+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:11.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:11.457+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:11.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:11.457+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:11.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:11.457+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:11.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:11 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:11.457+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:11.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:39:11.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:11.968 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:39:11.968 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:39:11.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:39:12.293 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 systemd[1]: Stopping Ceph osd.8 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:12.293 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:12.182+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:12.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:12.182+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:12.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:12.182+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:12.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:12.182+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:12.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:12 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:12.233+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:12.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:12 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:12.233+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:12.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:12 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:12.233+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:12.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:12 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:12.233+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:12.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:12 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:12.233+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:12.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.297+0000 7fc8e9d41640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:12.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.297+0000 7fc8e9d41640 -1 osd.8 108 *** Got signal Terminated *** 2024-09-12T03:39:12.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.297+0000 7fc8e9d41640 -1 osd.8 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:39:12.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.352+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:12.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.352+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:12.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.352+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:12.794 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:12 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:12.352+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:12.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:12 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:12.481+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:12.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:12 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:12.481+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:12.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:12 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:12.481+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:12.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:12 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:12.481+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:12.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:12 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:12.481+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:13.505 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:13 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:13.263+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:13.505 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:13 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:13.263+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:13.505 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:13 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:13.263+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:13.505 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:13 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:13.263+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:13.505 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:13 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:13.263+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:13.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:13.369+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:13.369+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:13.369+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:13.369+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:13.161+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:13.161+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:13.161+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:13.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:13 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:13.161+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:13.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:13.469+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:13.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:13.469+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:13.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:13.469+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:13.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:13.469+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:13.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:13 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:13.469+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:14.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:14.348+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:14.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:14.348+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:14.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:14.348+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:14.348+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:14.348+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:49.382871+0000 front 2024-09-12T03:38:49.382829+0000 (oldest deadline 2024-09-12T03:39:14.082928+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:14.164+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:14.164+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:14.164+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:14.164+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:14.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:14.164+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:14.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:14 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:14.282+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:14.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:14 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:14.282+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:14.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:14 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:14.282+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:14.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:14 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:14.282+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:14.606 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:14 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:14.282+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:14.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:14.440+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:14.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:14.440+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:14.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:14.440+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:14.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:14.440+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:14.665 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:14 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:14.440+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:15.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:15.318+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:15.318+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:15.318+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:15.318+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:15.318+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:49.382871+0000 front 2024-09-12T03:38:49.382829+0000 (oldest deadline 2024-09-12T03:39:14.082928+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:15.124+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:15.124+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:15.124+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:15.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:15.124+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:15.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:15 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:15.124+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:15.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:15.302+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:15.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:15.302+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:15.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:15.302+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:15.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:15.302+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:15.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:15 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:15.302+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:15.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:15 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:15.456+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:15.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:15 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:15.456+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:15.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:15 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:15.456+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:15.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:15 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:15.456+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:15.916 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:15 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:15.456+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:16.543 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:16.307+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:16.307+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:16.307+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:16.307+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:16.307+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:49.382871+0000 front 2024-09-12T03:38:49.382829+0000 (oldest deadline 2024-09-12T03:39:14.082928+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:16.170+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:16.170+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:16.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:16.170+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:16.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:16.170+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:16.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:16 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:16.170+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:16.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:16 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:16.268+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:16.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:16 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:16.268+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:16.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:16 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:16.268+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:16.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:16 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:16.268+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:16.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:16 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:16.268+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:16 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:16.411+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:16 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:16.411+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:16 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:16.411+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:16 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:16.411+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:16 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:16.411+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:16.664 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:16 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:16.411+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:17.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:17.261+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.075271+0000 front 2024-09-12T03:38:10.075445+0000 (oldest deadline 2024-09-12T03:38:34.175191+0000) 2024-09-12T03:39:17.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:17.261+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.976723+0000 front 2024-09-12T03:38:19.976951+0000 (oldest deadline 2024-09-12T03:38:41.676843+0000) 2024-09-12T03:39:17.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:17.261+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.378932+0000 front 2024-09-12T03:38:27.378970+0000 (oldest deadline 2024-09-12T03:38:49.679063+0000) 2024-09-12T03:39:17.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:17.261+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.180003+0000 front 2024-09-12T03:38:33.180189+0000 (oldest deadline 2024-09-12T03:38:59.079947+0000) 2024-09-12T03:39:17.544 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8[54138]: 2024-09-12T03:39:17.261+0000 7fc8e234f640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:49.382871+0000 front 2024-09-12T03:38:49.382829+0000 (oldest deadline 2024-09-12T03:39:14.082928+0000) 2024-09-12T03:39:17.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:17.195+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:17.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:17.195+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:17.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:17.195+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:17.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:17.195+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:17.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:17 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:17.195+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:17.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:17.243+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:17.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:17.243+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:17.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:17.243+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:17.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:17.243+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:17.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:17.243+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:17.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:17 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:17.243+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:17.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:17.444+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:17.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:17.444+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:17.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:17.444+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:17.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:17.444+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:17.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:17.444+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:17.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:17 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:17.444+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:18.041 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:17 smithi142 podman[72942]: 2024-09-12 03:39:17.740277873 +0000 UTC m=+5.569948486 container died 773052ed70f5fc7155dcb3a9f3d0e4a83081a798d2015ed26fcb653d29b02ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:39:18.293 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:18 smithi142 podman[72942]: 2024-09-12 03:39:18.139859273 +0000 UTC m=+5.969529887 container cleanup 773052ed70f5fc7155dcb3a9f3d0e4a83081a798d2015ed26fcb653d29b02ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:39:18.294 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:18 smithi142 bash[72942]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:39:18.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:18.187+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:18.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:18.187+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:18.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:18.187+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:18.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:18.187+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:18.294 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:18.187+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:18.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:18.231+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:18.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:18.231+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:18.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:18.231+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:18.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:18.231+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:18.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:18.231+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:18.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:18 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:18.231+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:18.793 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:18 smithi142 podman[72969]: 2024-09-12 03:39:18.357694311 +0000 UTC m=+0.612217448 container remove 773052ed70f5fc7155dcb3a9f3d0e4a83081a798d2015ed26fcb653d29b02ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T03:39:18.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:18 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:18.471+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:18.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:18 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:18.471+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:18.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:18 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:18.471+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:18.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:18 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:18.471+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:18.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:18 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:18.471+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:18.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:18 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:18.471+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:19.069 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:18 smithi142 podman[73060]: 2024-09-12 03:39:18.703993075 +0000 UTC m=+0.015968965 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:39:19.069 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:18 smithi142 podman[73060]: 2024-09-12 03:39:18.808361818 +0000 UTC m=+0.120337690 container create 8fb17f9ddeb28f37e2c009424eb1d34a07f8ee0476ed922ad7568a8cb86406e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:39:19.069 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:19 smithi142 podman[73060]: 2024-09-12 03:39:19.069351874 +0000 UTC m=+0.381327746 container init 8fb17f9ddeb28f37e2c009424eb1d34a07f8ee0476ed922ad7568a8cb86406e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T03:39:19.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:19 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:19.208+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:19.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:19 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:19.208+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:19.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:19 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:19.208+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:19.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:19 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:19.208+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:19.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:19 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:19.208+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:19.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:19 smithi142 podman[73060]: 2024-09-12 03:39:19.072920631 +0000 UTC m=+0.384896504 container start 8fb17f9ddeb28f37e2c009424eb1d34a07f8ee0476ed922ad7568a8cb86406e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:39:19.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:19 smithi142 podman[73060]: 2024-09-12 03:39:19.152742561 +0000 UTC m=+0.464718451 container attach 8fb17f9ddeb28f37e2c009424eb1d34a07f8ee0476ed922ad7568a8cb86406e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:39:19.545 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 12 03:39:19 smithi142 podman[73060]: 2024-09-12 03:39:19.269714168 +0000 UTC m=+0.581690045 container died 8fb17f9ddeb28f37e2c009424eb1d34a07f8ee0476ed922ad7568a8cb86406e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:39:19.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:19.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:19.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:19.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:19.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:19.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:19.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:19.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:19.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:19.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:19.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:19 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:19.237+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:19.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:19.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:19.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:19.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:19.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:19.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:19.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:19.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:19.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:19.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:19.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:19 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:19.515+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:19.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:39:20.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:20.018 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:39:20.018 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:39:20.018 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:39:20.414 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 systemd[1]: Stopping Ceph osd.9 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:20.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:20 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:20.217+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:20.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:20 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:20.217+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:20.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:20 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:20.217+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:20.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:20 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:20.217+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:20.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:20 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:20.217+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:20.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:20.212+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:20.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:20.212+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:20.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:20.212+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:20.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:20.212+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:20.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:20.212+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:20.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:20 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:20.212+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:20.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.431+0000 7f8a63e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:20.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.431+0000 7f8a63e00640 -1 osd.9 108 *** Got signal Terminated *** 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.431+0000 7f8a63e00640 -1 osd.9 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.561+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.561+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.561+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.561+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.561+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:20.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:20 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:20.561+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:21.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:21 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:21.188+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:21.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:21 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:21.188+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:21.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:21 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:21.188+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:21.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:21 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:21.188+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:21.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:21 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:21.188+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:21.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:21.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:21.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:21.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:21.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:21.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:21.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:21.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:21.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:21.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:21.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:21 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:21.163+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:21.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:21.569+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:21.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:21.569+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:21.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:21.569+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:21.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:21.569+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:21.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:21.569+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:21.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:21 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:21.569+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:22.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:22.194+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:22.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:22.194+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:22.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:22.194+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:22.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:22.194+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:22.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:22.194+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:22.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:22.161+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:22.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:22.161+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:22.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:22.161+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:22.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:22.161+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:22.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:22.161+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:22.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:22 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:22.161+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:22.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:22 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:22.550+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:22.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:22 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:22.550+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:22.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:22 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:22.550+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:22.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:22 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:22.550+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:22.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:22 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:22.550+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:22.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:22 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:22.550+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:23.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:23.153+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:23.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:23.153+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:23.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:23.153+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:23.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:23.153+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:23.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:23 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:23.153+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:23.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:23 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:23.175+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:23.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:23 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:23.175+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:23.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:23 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:23.175+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:23.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:23 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:23.175+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:23.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:23 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:23.175+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:23.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:23 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:23.175+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:23.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:23 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:23.548+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:23.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:23 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:23.548+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:23.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:23 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:23.548+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:23.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:23 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:23.548+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:23.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:23 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:23.548+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:23.915 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:23 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:23.548+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:24.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:24 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:24.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:24.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:24 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:24.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:24.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:24 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:24.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:24.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:24 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:24.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:24.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:24 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:24.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:24.558 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:24.559 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:24.559 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:24.559 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:24.559 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:24.559 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:24.559 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:24 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:24.208+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:24.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:24 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:24.577+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:10.214726+0000 front 2024-09-12T03:38:10.214757+0000 (oldest deadline 2024-09-12T03:38:34.914914+0000) 2024-09-12T03:39:24.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:24 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:24.577+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:19.417127+0000 front 2024-09-12T03:38:19.417392+0000 (oldest deadline 2024-09-12T03:38:45.317004+0000) 2024-09-12T03:39:24.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:24 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:24.577+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:26.919501+0000 front 2024-09-12T03:38:26.919447+0000 (oldest deadline 2024-09-12T03:38:51.019651+0000) 2024-09-12T03:39:24.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:24 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:24.577+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:33.921153+0000 front 2024-09-12T03:38:33.921181+0000 (oldest deadline 2024-09-12T03:38:59.221318+0000) 2024-09-12T03:39:24.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:24 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:24.577+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:39.221851+0000 front 2024-09-12T03:38:39.221820+0000 (oldest deadline 2024-09-12T03:39:09.222622+0000) 2024-09-12T03:39:24.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:24 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9[64603]: 2024-09-12T03:39:24.577+0000 7f8a5f800640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.524159+0000 front 2024-09-12T03:38:51.524293+0000 (oldest deadline 2024-09-12T03:39:15.624139+0000) 2024-09-12T03:39:25.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:25 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:25.075+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:25.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:25 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:25.075+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:25.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:25 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:25.075+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:25.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:25 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:25.075+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:25.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:25 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:25.075+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:25.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:25 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:25.075+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:25.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:25 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:25.255+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:26.213 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:25 smithi050 podman[111615]: 2024-09-12 03:39:25.944414828 +0000 UTC m=+5.622528538 container died 339f5bb59d820c2b98d680eb6ae65aa8c291012aba6e9556dc4e8ce1f05b7f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:39:26.479 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:26 smithi050 podman[111615]: 2024-09-12 03:39:26.320531525 +0000 UTC m=+5.998645231 container cleanup 339f5bb59d820c2b98d680eb6ae65aa8c291012aba6e9556dc4e8ce1f05b7f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T03:39:26.480 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:26 smithi050 bash[111615]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:39:26.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:26.077+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:26.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:26.077+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:26.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:26.077+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:26.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:26.077+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:26.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:26.077+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:26.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:26.077+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:26.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:26.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:26.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:26.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:26.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:26.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:26.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:26 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:26.245+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:26.914 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:26 smithi050 podman[111633]: 2024-09-12 03:39:26.479896049 +0000 UTC m=+0.523322884 container remove 339f5bb59d820c2b98d680eb6ae65aa8c291012aba6e9556dc4e8ce1f05b7f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:39:27.308 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:27 smithi050 podman[111719]: 2024-09-12 03:39:26.925941372 +0000 UTC m=+0.028680855 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:39:27.308 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:27 smithi050 podman[111719]: 2024-09-12 03:39:27.039299428 +0000 UTC m=+0.142038893 container create a8ddf2624b6d6b2d989fe13b240bf314df459ac49e907c76c5b9346aa76e594c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:39:27.309 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:27 smithi050 podman[111719]: 2024-09-12 03:39:27.309002206 +0000 UTC m=+0.411741696 container init a8ddf2624b6d6b2d989fe13b240bf314df459ac49e907c76c5b9346aa76e594c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:39:27.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:27 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:27.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:27.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:27 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:27.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:27.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:27 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:27.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:27.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:27 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:27.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:27.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:27 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:27.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:27.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:27 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:27.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:27.577 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:27 smithi050 podman[111719]: 2024-09-12 03:39:27.315919653 +0000 UTC m=+0.418659116 container start a8ddf2624b6d6b2d989fe13b240bf314df459ac49e907c76c5b9346aa76e594c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:39:27.578 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:27 smithi050 podman[111719]: 2024-09-12 03:39:27.383763112 +0000 UTC m=+0.486502592 container attach a8ddf2624b6d6b2d989fe13b240bf314df459ac49e907c76c5b9346aa76e594c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:39:27.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:27.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:27.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:27.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:27.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:27.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:27.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:27 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:27.210+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:27.895 INFO:journalctl@ceph.osd.9.smithi050.stdout:Sep 12 03:39:27 smithi050 podman[111719]: 2024-09-12 03:39:27.578161965 +0000 UTC m=+0.680901449 container died a8ddf2624b6d6b2d989fe13b240bf314df459ac49e907c76c5b9346aa76e594c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:39:28.232 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:39:28.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:28.308 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:39:28.308 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:39:28.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:39:28.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:28 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:28.168+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:28.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:28 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:28.168+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:28.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:28 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:28.168+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:28.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:28 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:28.168+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:28.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:28 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:28.168+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:28.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:28 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:28.168+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:28.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:28.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:28.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:28.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:28.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:28.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:28.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.187+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:28.605 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 systemd[1]: Stopping Ceph osd.10 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:29.104 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.624+0000 7f1e2ca0e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:29.104 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.624+0000 7f1e2ca0e640 -1 osd.10 108 *** Got signal Terminated *** 2024-09-12T03:39:29.104 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:28 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:28.624+0000 7f1e2ca0e640 -1 osd.10 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:39:29.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:29 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:29.127+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:29.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:29 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:29.127+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:29.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:29 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:29.127+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:29.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:29 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:29.127+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:29.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:29 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:29.127+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:29.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:29 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:29.127+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:29.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:29 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:29.142+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:30.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:30.083+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:30.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:30.083+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:30.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:30.083+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:30.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:30.083+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:30.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:30.083+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:30.545 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:30 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:30.083+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:30.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:30 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:30.112+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:31.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:31 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:31.080+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:31.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:31.081+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:31.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:31.081+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:31.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:31.081+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:31.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:31.081+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:31.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:31.081+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:31.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:31 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:31.081+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:32.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:32.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:32.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:32.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:32.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:32.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:32.355 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:32 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:32.039+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:32.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:32.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:32 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:32.115+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:11.827490+0000 front 2024-09-12T03:38:11.827442+0000 (oldest deadline 2024-09-12T03:38:32.927184+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.728474+0000 front 2024-09-12T03:38:18.728578+0000 (oldest deadline 2024-09-12T03:38:41.028692+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:27.430125+0000 front 2024-09-12T03:38:27.430063+0000 (oldest deadline 2024-09-12T03:38:52.730109+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-09-12T03:38:32.730794+0000 front 2024-09-12T03:38:32.730664+0000 (oldest deadline 2024-09-12T03:38:57.331130+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:43.732503+0000 front 2024-09-12T03:38:43.732546+0000 (oldest deadline 2024-09-12T03:39:07.232719+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:51.234061+0000 front 2024-09-12T03:38:51.234158+0000 (oldest deadline 2024-09-12T03:39:17.133968+0000) 2024-09-12T03:39:33.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:33 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10[59235]: 2024-09-12T03:39:33.077+0000 7f1e2501c640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:38:59.335337+0000 front 2024-09-12T03:38:59.335509+0000 (oldest deadline 2024-09-12T03:39:23.434986+0000) 2024-09-12T03:39:33.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:33.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:33.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:33.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:33.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:33.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:33.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:33 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:33.150+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:34.354 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:34 smithi082 podman[71406]: 2024-09-12 03:39:34.085119774 +0000 UTC m=+5.581992696 container died a2c210e9e9a8c475f7dc05b8c1d6ff2e4b29ea1dc9c89ad9082af86a01d50d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:39:34.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:34.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:34.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:34.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:34.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:34.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:34.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:34 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:34.126+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:34.716 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:34 smithi082 podman[71406]: 2024-09-12 03:39:34.448931289 +0000 UTC m=+5.945804204 container cleanup a2c210e9e9a8c475f7dc05b8c1d6ff2e4b29ea1dc9c89ad9082af86a01d50d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:39:34.716 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:34 smithi082 bash[71406]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:39:35.033 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:34 smithi082 podman[71420]: 2024-09-12 03:39:34.71658651 +0000 UTC m=+0.624730135 container remove a2c210e9e9a8c475f7dc05b8c1d6ff2e4b29ea1dc9c89ad9082af86a01d50d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:39:35.327 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:35 smithi082 podman[71511]: 2024-09-12 03:39:35.03378287 +0000 UTC m=+0.093188410 container create 1abf59fcd3bb9e2ff5aa628bcf07eb63f074b3339b67330c5fbee7e0e9fff3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:39:35.327 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:35 smithi082 podman[71511]: 2024-09-12 03:39:34.956538762 +0000 UTC m=+0.015944312 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:39:35.327 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:35 smithi082 podman[71511]: 2024-09-12 03:39:35.252636046 +0000 UTC m=+0.312041586 container init 1abf59fcd3bb9e2ff5aa628bcf07eb63f074b3339b67330c5fbee7e0e9fff3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:39:35.327 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:35 smithi082 podman[71511]: 2024-09-12 03:39:35.255786772 +0000 UTC m=+0.315192317 container start 1abf59fcd3bb9e2ff5aa628bcf07eb63f074b3339b67330c5fbee7e0e9fff3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:39:35.327 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:35 smithi082 podman[71511]: 2024-09-12 03:39:35.327509988 +0000 UTC m=+0.386915546 container attach 1abf59fcd3bb9e2ff5aa628bcf07eb63f074b3339b67330c5fbee7e0e9fff3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:35.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:35 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:35.151+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:35.604 INFO:journalctl@ceph.osd.10.smithi082.stdout:Sep 12 03:39:35 smithi082 podman[71511]: 2024-09-12 03:39:35.444626415 +0000 UTC m=+0.504031968 container died 1abf59fcd3bb9e2ff5aa628bcf07eb63f074b3339b67330c5fbee7e0e9fff3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-12T03:39:36.082 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:39:36.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:36.122 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:39:36.122 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:39:36.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:39:36.441 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.110+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.111+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 systemd[1]: Stopping Ceph osd.11 for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.441+0000 7f8cac1ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.441+0000 7f8cac1ae640 -1 osd.11 108 *** Got signal Terminated *** 2024-09-12T03:39:36.442 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:36 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:36.441+0000 7f8cac1ae640 -1 osd.11 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:39:37.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:37.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:37.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:37.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:37.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:37.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:37.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:37 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:37.109+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:38.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:38.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:38.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:38.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:38.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:38.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:38.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:38 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:38.119+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:39.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:39.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:39.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:39.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:39.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:39.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:39.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:39 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:39.130+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:40.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:40.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:40.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:40.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:40.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:40.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:40.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:40 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:40.156+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:41.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-09-12T03:38:09.543205+0000 front 2024-09-12T03:38:09.543192+0000 (oldest deadline 2024-09-12T03:38:33.043017+0000) 2024-09-12T03:39:41.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-09-12T03:38:18.944393+0000 front 2024-09-12T03:38:18.944212+0000 (oldest deadline 2024-09-12T03:38:43.044348+0000) 2024-09-12T03:39:41.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-12T03:38:23.044750+0000 front 2024-09-12T03:38:23.044779+0000 (oldest deadline 2024-09-12T03:38:48.944957+0000) 2024-09-12T03:39:41.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-09-12T03:38:40.547684+0000 front 2024-09-12T03:38:40.547508+0000 (oldest deadline 2024-09-12T03:39:08.748178+0000) 2024-09-12T03:39:41.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-12T03:38:48.748709+0000 front 2024-09-12T03:38:48.748806+0000 (oldest deadline 2024-09-12T03:39:14.048733+0000) 2024-09-12T03:39:41.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-09-12T03:39:00.450334+0000 front 2024-09-12T03:39:00.450374+0000 (oldest deadline 2024-09-12T03:39:24.550014+0000) 2024-09-12T03:39:41.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:41 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11[60682]: 2024-09-12T03:39:41.205+0000 7f8ca87c4640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-09-12T03:39:07.450856+0000 front 2024-09-12T03:39:07.450941+0000 (oldest deadline 2024-09-12T03:39:31.550850+0000) 2024-09-12T03:39:42.291 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:42 smithi142 podman[73162]: 2024-09-12 03:39:42.019301384 +0000 UTC m=+5.705427971 container died cff3cd68b597bed6b82b093cff6b2229797acd7014742572deae6c8ed61246e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T03:39:42.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:42 smithi142 podman[73162]: 2024-09-12 03:39:42.391182389 +0000 UTC m=+6.077309021 container cleanup cff3cd68b597bed6b82b093cff6b2229797acd7014742572deae6c8ed61246e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T03:39:42.544 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:42 smithi142 bash[73162]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:39:42.962 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:42 smithi142 podman[73180]: 2024-09-12 03:39:42.642217668 +0000 UTC m=+0.617037106 container remove cff3cd68b597bed6b82b093cff6b2229797acd7014742572deae6c8ed61246e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:39:43.270 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:42 smithi142 podman[73267]: 2024-09-12 03:39:42.865349077 +0000 UTC m=+0.015921113 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:39:43.270 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:42 smithi142 podman[73267]: 2024-09-12 03:39:42.967786177 +0000 UTC m=+0.118358211 container create 9c7122bd7f6ddf4aa08fdc4637ea30843e9ee83f88103072b9431c0c2ab62c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T03:39:43.270 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:43 smithi142 podman[73267]: 2024-09-12 03:39:43.187070549 +0000 UTC m=+0.337642577 container init 9c7122bd7f6ddf4aa08fdc4637ea30843e9ee83f88103072b9431c0c2ab62c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:39:43.270 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:43 smithi142 podman[73267]: 2024-09-12 03:39:43.190489801 +0000 UTC m=+0.341061830 container start 9c7122bd7f6ddf4aa08fdc4637ea30843e9ee83f88103072b9431c0c2ab62c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:39:43.270 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:43 smithi142 podman[73267]: 2024-09-12 03:39:43.270323412 +0000 UTC m=+0.420895443 container attach 9c7122bd7f6ddf4aa08fdc4637ea30843e9ee83f88103072b9431c0c2ab62c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T03:39:43.543 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 12 03:39:43 smithi142 podman[73267]: 2024-09-12 03:39:43.38069024 +0000 UTC m=+0.531262269 container died 9c7122bd7f6ddf4aa08fdc4637ea30843e9ee83f88103072b9431c0c2ab62c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:39:44.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:39:44.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:44.086 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:39:44.086 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:39:44.086 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:39:44.414 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:44 smithi050 systemd[1]: Stopping Ceph mds.a for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:44.914 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a[71468]: 2024-09-12T03:39:44.483+0000 7fc1f2000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:44.914 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:44 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a[71468]: 2024-09-12T03:39:44.483+0000 7fc1f2000640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:39:45.825 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:45 smithi050 podman[111846]: 2024-09-12 03:39:45.553064317 +0000 UTC m=+1.182043105 container died 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:39:46.099 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:45 smithi050 podman[111846]: 2024-09-12 03:39:45.91454576 +0000 UTC m=+1.543524548 container cleanup 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:39:46.099 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:45 smithi050 bash[111846]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:39:46.353 INFO:journalctl@ceph.mds.a.smithi050.stdout:Sep 12 03:39:46 smithi050 podman[111858]: 2024-09-12 03:39:46.099071427 +0000 UTC m=+0.533238027 container remove 2c1f08c25108f3047cd6f204a9db324c3274ae8fc7499a9981aebf7c01e6b4b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:39:46.362 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:39:46.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:46.435 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:39:46.435 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:39:46.435 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:39:46.664 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:46 smithi050 systemd[1]: Stopping Ceph mds.d for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:47.164 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:46 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d[70299]: 2024-09-12T03:39:46.829+0000 7f0732400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:47.164 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:46 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d[70299]: 2024-09-12T03:39:46.829+0000 7f0732400640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:39:49.310 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:49 smithi050 podman[111966]: 2024-09-12 03:39:49.052587052 +0000 UTC m=+2.337165057 container died be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:39:49.594 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:49 smithi050 podman[111966]: 2024-09-12 03:39:49.417957134 +0000 UTC m=+2.702535138 container cleanup be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:39:49.594 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:49 smithi050 bash[111966]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:39:49.862 INFO:journalctl@ceph.mds.d.smithi050.stdout:Sep 12 03:39:49 smithi050 podman[111978]: 2024-09-12 03:39:49.594493589 +0000 UTC m=+0.533251241 container remove be85d7fc58f65d5e5b38d0ae77e3c690956f69953f8676f7e23d27c7f2c974dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:39:49.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:39:49.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:49.941 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:39:49.941 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:39:49.941 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:39:50.164 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:50 smithi050 systemd[1]: Stopping Ceph mds.g for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:50.664 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g[71099]: 2024-09-12T03:39:50.324+0000 7fa639200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:50.664 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:50 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g[71099]: 2024-09-12T03:39:50.324+0000 7fa639200640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:39:53.823 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:53 smithi050 podman[112085]: 2024-09-12 03:39:53.560675141 +0000 UTC m=+3.347592010 container died 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T03:39:54.115 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:53 smithi050 podman[112085]: 2024-09-12 03:39:53.93068867 +0000 UTC m=+3.717605538 container cleanup 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:39:54.115 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:53 smithi050 bash[112085]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:39:54.369 INFO:journalctl@ceph.mds.g.smithi050.stdout:Sep 12 03:39:54 smithi050 podman[112098]: 2024-09-12 03:39:54.115119832 +0000 UTC m=+0.540035073 container remove 619790d41b10365466cc61fe4f82fc6048620f1861938ba62c969d15ad426518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:39:54.378 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:39:54.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:54.448 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:39:54.448 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:39:54.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:39:54.664 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:54 smithi050 systemd[1]: Stopping Ceph mds.j for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:55.164 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j[69917]: 2024-09-12T03:39:54.828+0000 7f5a08a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:55.164 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:54 smithi050 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j[69917]: 2024-09-12T03:39:54.828+0000 7f5a08a00640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:39:57.410 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:57 smithi050 podman[112206]: 2024-09-12 03:39:57.138885419 +0000 UTC m=+2.416209715 container died c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-12T03:39:57.664 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:57 smithi050 podman[112206]: 2024-09-12 03:39:57.525850933 +0000 UTC m=+2.803175231 container cleanup c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:39:57.664 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:57 smithi050 bash[112206]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:39:58.057 INFO:journalctl@ceph.mds.j.smithi050.stdout:Sep 12 03:39:57 smithi050 podman[112218]: 2024-09-12 03:39:57.885565336 +0000 UTC m=+0.731635432 container remove c76b3d8286d28860143ac0db2d7f4ce31952cf1c62e0d70fe4db7eaf52791a02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:39:58.092 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:39:58.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:39:58.161 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:39:58.161 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:39:58.161 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:39:58.469 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:39:58 smithi082 systemd[1]: Stopping Ceph mds.b for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:39:58.734 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:39:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: 2024-09-12T03:39:58.468+0000 7effb04eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:39:58.735 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:39:58 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b[64877]: 2024-09-12T03:39:58.468+0000 7effb04eb640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:40:01.746 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:40:01 smithi082 podman[71612]: 2024-09-12 03:40:01.485300818 +0000 UTC m=+3.126846338 container died 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:40:02.104 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:40:01 smithi082 podman[71612]: 2024-09-12 03:40:01.829394594 +0000 UTC m=+3.470940155 container cleanup 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T03:40:02.104 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:40:01 smithi082 bash[71612]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:40:02.104 INFO:journalctl@ceph.mds.b.smithi082.stdout:Sep 12 03:40:01 smithi082 podman[71637]: 2024-09-12 03:40:01.988844936 +0000 UTC m=+0.497163405 container remove 5490d721def5742bdea4c04bb8c33306d824e47768c2c930d1cdbb1d9aa948c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:40:02.218 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:40:02.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:02.253 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:40:02.254 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:40:02.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:40:02.572 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:40:02 smithi082 systemd[1]: Stopping Ceph mds.e for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:02.854 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:40:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e[64264]: 2024-09-12T03:40:02.572+0000 7feeea3bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:02.854 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:40:02 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e[64264]: 2024-09-12T03:40:02.572+0000 7feeea3bc640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:40:05.757 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:40:05 smithi082 podman[71747]: 2024-09-12 03:40:05.487636696 +0000 UTC m=+3.003277090 container died ec81a90b9220f59ca8418989c7b5296da8f5ca37b52fd0077114ea3f1e61c445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:40:06.104 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:40:05 smithi082 podman[71747]: 2024-09-12 03:40:05.84101666 +0000 UTC m=+3.356657053 container cleanup ec81a90b9220f59ca8418989c7b5296da8f5ca37b52fd0077114ea3f1e61c445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T03:40:06.104 INFO:journalctl@ceph.mds.e.smithi082.stdout:Sep 12 03:40:05 smithi082 bash[71747]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:40:06.263 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:40:06.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:06.298 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:40:06.298 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:40:06.298 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:40:06.854 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:40:06 smithi082 systemd[1]: Stopping Ceph mds.h for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:06.854 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:40:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h[64574]: 2024-09-12T03:40:06.619+0000 7f365d0bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:06.854 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:40:06 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h[64574]: 2024-09-12T03:40:06.619+0000 7f365d0bc640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:40:09.744 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:40:09 smithi082 podman[71867]: 2024-09-12 03:40:09.488436186 +0000 UTC m=+2.969892606 container died fa8492535347c21f2cff0a2ad6cc88c7f1a016f62fb15bc32d3effab74b880eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:40:10.077 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:40:09 smithi082 podman[71867]: 2024-09-12 03:40:09.835818948 +0000 UTC m=+3.317275367 container cleanup fa8492535347c21f2cff0a2ad6cc88c7f1a016f62fb15bc32d3effab74b880eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:40:10.078 INFO:journalctl@ceph.mds.h.smithi082.stdout:Sep 12 03:40:09 smithi082 bash[71867]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:40:10.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:40:10.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:10.279 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:40:10.279 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:40:10.279 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:40:10.604 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:40:10 smithi082 systemd[1]: Stopping Ceph mds.k for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:10.604 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:40:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k[63950]: 2024-09-12T03:40:10.581+0000 7fe21d108640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:10.604 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:40:10 smithi082 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k[63950]: 2024-09-12T03:40:10.581+0000 7fe21d108640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:40:13.597 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:40:13 smithi082 podman[71989]: 2024-09-12 03:40:13.327379226 +0000 UTC m=+2.834040602 container died 53d8cbcad9db2105efc95b49502400b083094632e7b3e4c13071ac16abe0a36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:40:13.854 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:40:13 smithi082 podman[71989]: 2024-09-12 03:40:13.697588698 +0000 UTC m=+3.204250116 container cleanup 53d8cbcad9db2105efc95b49502400b083094632e7b3e4c13071ac16abe0a36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:40:13.854 INFO:journalctl@ceph.mds.k.smithi082.stdout:Sep 12 03:40:13 smithi082 bash[71989]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:40:14.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:40:14.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:14.105 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:40:14.105 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:40:14.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:40:14.474 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:40:14 smithi142 systemd[1]: Stopping Ceph mds.c for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:14.734 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:40:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c[65834]: 2024-09-12T03:40:14.473+0000 7f17eb73d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:14.735 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:40:14 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c[65834]: 2024-09-12T03:40:14.473+0000 7f17eb73d640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:40:17.793 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:40:17 smithi142 podman[73383]: 2024-09-12 03:40:17.496178202 +0000 UTC m=+3.199847636 container died 6b794d63593af548cdf307ff3db49df87b49403353e14a0ef86a70498d25c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:40:18.265 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:40:17 smithi142 podman[73383]: 2024-09-12 03:40:17.998169756 +0000 UTC m=+3.701839190 container cleanup 6b794d63593af548cdf307ff3db49df87b49403353e14a0ef86a70498d25c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T03:40:18.265 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 12 03:40:17 smithi142 bash[73383]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:40:18.425 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:40:18.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:18.462 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:40:18.462 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:40:18.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:40:18.793 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:40:18 smithi142 systemd[1]: Stopping Ceph mds.f for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:19.293 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:40:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[69849]: 2024-09-12T03:40:18.793+0000 7fc1ff53e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:19.293 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:40:18 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f[69849]: 2024-09-12T03:40:18.793+0000 7fc1ff53e640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:40:21.769 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:40:21 smithi142 podman[73515]: 2024-09-12 03:40:21.496812661 +0000 UTC m=+2.807497831 container died 58bf2d33b166973f068bc9e1dfab874142d4127fb4d52b54804ce8165dd60827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T03:40:22.043 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:40:21 smithi142 podman[73515]: 2024-09-12 03:40:21.877061801 +0000 UTC m=+3.187746973 container cleanup 58bf2d33b166973f068bc9e1dfab874142d4127fb4d52b54804ce8165dd60827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:40:22.043 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 12 03:40:21 smithi142 bash[73515]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:40:22.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:40:22.289 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:22.289 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:40:22.289 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:40:22.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:40:22.630 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:40:22 smithi142 systemd[1]: Stopping Ceph mds.i for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:23.043 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:40:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i[66137]: 2024-09-12T03:40:22.630+0000 7f2fee80a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:23.043 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:40:22 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i[66137]: 2024-09-12T03:40:22.630+0000 7f2fee80a640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:40:25.755 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:40:25 smithi142 podman[73638]: 2024-09-12 03:40:25.494031115 +0000 UTC m=+2.979468372 container died 6a9a1ad592c82259699d07d1ca4f4bffecc430699be75dd6fb5ff5fada353217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T03:40:26.043 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:40:25 smithi142 podman[73638]: 2024-09-12 03:40:25.85550051 +0000 UTC m=+3.340937722 container cleanup 6a9a1ad592c82259699d07d1ca4f4bffecc430699be75dd6fb5ff5fada353217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:40:26.043 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 12 03:40:25 smithi142 bash[73638]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:40:26.344 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:40:26.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:26.377 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:40:26.377 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:40:26.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:40:26.704 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:40:26 smithi142 systemd[1]: Stopping Ceph mds.l for 389f22da-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:40:27.043 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:40:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l[65221]: 2024-09-12T03:40:26.701+0000 7fef3e072640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:40:27.043 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:40:26 smithi142 ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l[65221]: 2024-09-12T03:40:26.701+0000 7fef3e072640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:40:30.901 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:40:30 smithi142 podman[73761]: 2024-09-12 03:40:30.626443187 +0000 UTC m=+4.029254501 container died ad4f9db4c1cf488768538fb6efa891b8ebcd67a18863c19b374b05d0ac0cfd32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T03:40:31.293 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:40:30 smithi142 podman[73761]: 2024-09-12 03:40:30.98418559 +0000 UTC m=+4.386996891 container cleanup ad4f9db4c1cf488768538fb6efa891b8ebcd67a18863c19b374b05d0ac0cfd32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:40:31.293 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 12 03:40:30 smithi142 bash[73761]: ceph-389f22da-70b5-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:40:31.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-389f22da-70b5-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:40:31.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:40:31.586 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:40:31.587 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:40:31.872 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:40:35.811 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:40:36.014 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:40:38.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:40:38.656 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:40:40.980 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:40:41.045 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:40:41.078 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:40:41.105 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:40:41.107 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi050/crash 2024-09-12T03:40:41.108 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:40:41.174 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:40:41.174 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:40:41.177 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi082/crash 2024-09-12T03:40:41.178 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:40:41.295 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:40:41.295 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:40:41.297 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi142/crash 2024-09-12T03:40:41.298 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:40:41.338 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/389f22da-70b5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:40:41.338 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:40:41.339 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:40:41.340 DEBUG:teuthology.orchestra.run.smithi050:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-12T03:40:41.420 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:40:41.421 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:40:41.424 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:40:41.427 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:40:41.454 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:40:41.454 INFO:teuthology.orchestra.run.smithi142.stderr:: No such file or directory 2024-09-12T03:40:41.454 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:40:41.454 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:40:41.454 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:40:41.457 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:40:41.457 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:40:41.457 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:40:41.457 INFO:teuthology.orchestra.run.smithi142.stderr: 81.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:40:41.457 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:: No such file or directory 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:40:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:40:41.460 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:40:41.460 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.2.log: /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.5.log: /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:40:41.460 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:40:41.460 INFO:teuthology.orchestra.run.smithi142.stderr: 87.1% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:40:41.460 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:40:41.461 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:40:41.462 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:40:41.462 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:40:41.462 INFO:teuthology.orchestra.run.smithi082.stderr: 94.4% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:40:41.462 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:40:41.463 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:40:41.463 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:40:41.463 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.8.log: 94.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:40:41.465 INFO:teuthology.orchestra.run.smithi142.stderr: 91.0% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:40:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.y.log: /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log: 87.1% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:40:41.469 INFO:teuthology.orchestra.run.smithi082.stderr: 91.0% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:40:41.470 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:40:41.470 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:40:41.472 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:40:41.472 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:40:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:40:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:40:41.477 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-09-12T03:40:41.477 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-12T03:40:41.478 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:40:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:40:41.480 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:40:41.480 INFO:teuthology.orchestra.run.smithi050.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-12T03:40:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:40:41.482 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:40:41.482 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:40:41.483 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:40:41.483 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:40:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:40:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log: 84.3% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:40:41.487 INFO:teuthology.orchestra.run.smithi050.stderr: 90.3% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:40:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:40:41.488 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:40:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:40:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.6.log: 94.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:40:41.490 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:40:41.494 INFO:teuthology.orchestra.run.smithi050.stderr: 94.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:40:41.497 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:40:41.498 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:40:41.499 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:40:41.501 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.d.log: 87.1% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:40:41.501 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:40:41.501 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:40:41.504 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:40:41.505 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:40:41.506 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.x.log: 94.3% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:40:41.508 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:40:41.517 INFO:teuthology.orchestra.run.smithi050.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:40:41.898 INFO:teuthology.orchestra.run.smithi050.stderr: 90.1% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:40:41.905 INFO:teuthology.orchestra.run.smithi142.stderr: 91.7% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:40:42.033 INFO:teuthology.orchestra.run.smithi082.stderr: 91.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:40:42.242 INFO:teuthology.orchestra.run.smithi082.stderr: 94.7% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:40:42.486 INFO:teuthology.orchestra.run.smithi082.stderr: 95.0% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:40:42.783 INFO:teuthology.orchestra.run.smithi050.stderr: 90.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi142.stderr: 96.0% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:40:49.081 INFO:teuthology.orchestra.run.smithi142.stderr: 93.4% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:40:49.662 INFO:teuthology.orchestra.run.smithi142.stderr: 93.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:40:52.206 INFO:teuthology.orchestra.run.smithi082.stderr: 92.0% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:40:57.815 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:40:57.994 INFO:teuthology.orchestra.run.smithi142.stderr: 91.7% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:41:00.203 INFO:teuthology.orchestra.run.smithi050.stderr: 91.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:41:01.138 INFO:teuthology.orchestra.run.smithi082.stderr: 91.9% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:41:02.641 INFO:teuthology.orchestra.run.smithi082.stderr: 91.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:41:03.071 INFO:teuthology.orchestra.run.smithi050.stderr: 91.8% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:41:06.124 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:41:06.394 INFO:teuthology.orchestra.run.smithi050.stderr: 91.6% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:41:06.397 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-12T03:41:06.397 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m24.944s 2024-09-12T03:41:06.398 INFO:teuthology.orchestra.run.smithi050.stderr:user 1m17.162s 2024-09-12T03:41:06.398 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m6.184s 2024-09-12T03:41:08.027 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:41:09.624 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:41:09.626 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-09-12T03:41:09.626 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m28.183s 2024-09-12T03:41:09.626 INFO:teuthology.orchestra.run.smithi082.stderr:user 1m45.319s 2024-09-12T03:41:09.626 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m3.239s 2024-09-12T03:41:11.074 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:41:17.258 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/389f22da-70b5-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:41:17.260 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-09-12T03:41:17.260 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m35.818s 2024-09-12T03:41:17.260 INFO:teuthology.orchestra.run.smithi142.stderr:user 2m1.382s 2024-09-12T03:41:17.260 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m3.707s 2024-09-12T03:41:17.260 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:41:17.261 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi050/log 2024-09-12T03:41:17.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:41:29.474 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi082/log 2024-09-12T03:41:29.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:41:45.582 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi142/log 2024-09-12T03:41:45.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:42:04.030 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:42:04.030 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --force 2024-09-12T03:42:04.317 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:42:05.075 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --force 2024-09-12T03:42:05.279 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:42:05.684 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 389f22da-70b5-11ef-bcea-c7b262605968 --force 2024-09-12T03:42:05.884 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 389f22da-70b5-11ef-bcea-c7b262605968 2024-09-12T03:42:06.300 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:42:06.301 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:42:06.341 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:42:06.358 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:42:06.374 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:42:06.375 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:42:06.387 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:42:06.388 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:42:06.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:42:06.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:42:06.545 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:42:06.556 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:42:06.556 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:42:06.561 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:42:06.563 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:42:06.576 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:42:06.578 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:42:06.580 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:42:06.580 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-09-12T03:42:06.580 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -3001us[-2977us] +/- 61ms 2024-09-12T03:42:06.580 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +1531us[+1531us] +/- 61ms 2024-09-12T03:42:06.580 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +2164us[+2188us] +/- 86ms 2024-09-12T03:42:06.581 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:42:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:42:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-12T03:42:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 -2990us[-2952us] +/- 61ms 2024-09-12T03:42:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +1543us[+1543us] +/- 61ms 2024-09-12T03:42:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +2171us[+2209us] +/- 86ms 2024-09-12T03:42:06.582 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:42:06.593 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:42:06.601 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:42:06.601 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-09-12T03:42:06.601 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 -2990us[-2953us] +/- 61ms 2024-09-12T03:42:06.601 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +1537us[+1574us] +/- 61ms 2024-09-12T03:42:06.601 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +2148us[+2148us] +/- 86ms 2024-09-12T03:42:06.601 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:42:06.602 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:42:06.615 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:42:06.615 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:42:06.643 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:42:06.777 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:42:06.846 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:42:06.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:42:06.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:42:06.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:42:07.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:42:07.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:42:07.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:42:07.032 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:42:07.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:42:07.083 INFO:teuthology.task.internal:Duration was 2009.183623 seconds 2024-09-12T03:42:07.084 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:42:07.118 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:42:07.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:42:07.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:42:07.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:42:07.161 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:42:07.163 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:42:07.208 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:42:07.642 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:42:07.642 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-09-12T03:42:07.642 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:42:07.695 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-09-12T03:42:07.695 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:42:07.723 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T03:42:07.724 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:42:07.753 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:42:07.754 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:42:07.757 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:42:07.766 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:42:07.820 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:42:07.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:42:07.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:42:07.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:42:08.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:42:08.531 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:42:08.532 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:42:08.588 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:42:08.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:42:08.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:42:08.653 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:42:08.657 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:42:08.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:42:08.682 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-09-12T03:42:08.707 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-09-12T03:42:08.724 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-09-12T03:42:08.754 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:42:08.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:42:08.789 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:42:08.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:42:08.805 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:42:08.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:42:08.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:42:08.842 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:42:08.844 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi050 2024-09-12T03:42:08.844 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:42:08.924 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi082 2024-09-12T03:42:08.925 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:42:08.961 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901675/remote/smithi142 2024-09-12T03:42:08.962 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:42:08.998 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:42:08.999 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:42:09.003 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:42:09.005 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:42:09.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:42:09.061 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:42:09.061 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:42:09.085 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:42:09.085 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:42:09.089 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:42:09.091 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:42:09.104 INFO:teuthology.orchestra.run.smithi082.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:42 /home/ubuntu/cephtest 2024-09-12T03:42:09.107 INFO:teuthology.orchestra.run.smithi142.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:42 /home/ubuntu/cephtest 2024-09-12T03:42:09.121 INFO:teuthology.orchestra.run.smithi050.stdout: 393731 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:42 /home/ubuntu/cephtest 2024-09-12T03:42:09.123 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:42:09.134 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:42:09.210 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} duration: 2009.1836230754852 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:42:09.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:42:09.305 INFO:teuthology.run:pass